LCG Logo
LCG Certification Test suites


Last modified : 07/04/2004 (should be updated regularly)
LCG release 2.0 - EDG release 2.x
Authors:
TSTG group

Location of the software: LCG CVS repository http://isscvs.cern.ch:8180/cgi-bin/cvsweb.cgi/
Where to find this help page:
http://grid-deployment.web.cern.ch/grid-deployment/tstg/docs/LCG-Certification-help


Summary:

The recommended steps are in bold.

  1. Is TSTG middleware available on your machine ?
    1. Private (or local) install through the tarball
    2. Local install through the RPMs
    3. Native install
  2. TSTG Site configuration
  3. Site Testing.
    1. Global Site Testing
    2. Partial Site Testing
  4. TSTG middleware installation, from CVS sources.
  5. Some of the tests available through the edg-site-certification module.

TSTG release:

Private install using the tarball: The test suites and tools are currently available as tarballs from the LCG Testing Group web site.

To install the tstg-pro (production version) tar file stored in the tstg/tars directory:

$ cd <install-dir>
$ wget http://grid-deployment.web.cern.ch/grid-deployment/tstg/tars/tstg-pro.tgz
$ tar xzf tstg-pro.tgz
$ rm tstg-pro.tgz

Remark: the tstg-dev (development version) and tstg-old (previous production version) tar files are also available and can be installed in the same way.

Important: the correspondance between the LCG tag and the tstg-pro and -dev versions is given in this tags.txt file, together with the relevant CVS tags of the 2 modules below.

The TSTG release is made of 2 modules:

- edg-tests
- edg-site-certification

The edg-tests module provides configuration and stress tests. It also includes various utilities driving automated sequences of tests and presenting the results in a common fashion (HTML array). It requires a small text file, acting as a site configuration file, to be filled up.

The edg-site-certification module is packaged into the same tarball to allow for synchronization of the 2 modules. It contains many functionality tests used by the automated test suite (see MainScript).

Private install from the RPMs: both modules are also available as RPMs, built in sync. To install the production version from the files stored in the tstg/tars directory:

$ cd <install-dir>
$ curl -O http://grid-deployment.web.cern.ch/grid-deployment/tstg/tars/install_pro_RPMs
$ source ./install_pro_RPMs
$ rm install_pro_RPMs

Warning: this only works for sh/bash shells.
Remark: the dev and old RPM flavors are also available in the same place and can be installed in the same way.

Native install
: the latter module (edg-site-certification) is currently the only one available in the standard /opt/edg directory tree, since it
is distributed together with EDG and LCG releases and available on every User Interface (UI), via RPMs. The main commands are available in the standard $PATH.


TSTG Site Configuration:

The edg-tests package requires a site configuration file to be supplied into the opt/edg/tests/etc directory. Have a look to the template: test_site-LocalTB.template

For testing a new LCG1 site, the site administrator must create and customize a test_site-localTB.conf file from the available template:

$ cd <install-dir>/opt/edg/tests/etc
$ cp test_site-LocalTB.template test_site-LocalTB.conf
## Then edit the file and set the machine hostnames and parameters of your local site
$ emacs -nw test_site-LocalTB.conf

Partial Site Testing:

Prerequisite: Obtain a user proxy:

$ grid-proxy-init -valid <hours:minutes>

Try interactive testing of your site:

UI configuration Test :
$ <install-dir>/opt/edg/bin/run_test.sh -t UI_config
Site Configuration file: /opt/edg/tests/etc/test_site-LocalTB.conf
.Op. System services [ OK ]
.EDG Environment [ OK ]
.Installed Packages [ OK ]
.EDG_WL UI configuration [ OK ]
.User commands [ OK ]
The [-x] option will generate an HTML report as shown there.
$ <install-dir>/opt/edg/bin/run_test.sh -x -t UI_config
opt/edg/tests/tests/InstallConfig/UI/UI_config_test.def selected...
UI config running..........
Site Configuration file: opt/edg/tests/etc/test_site-LocalTB.conf
.....UI config done
Formatting output to html..........done
Html file saved: /tmp/031006_161412_UI_config/index.html
$ netscape /tmp/031006_161412_UI_config/index.html &
Site Testing : 
Run the following command. The [-x] option will generate an HTML report as shown there.
$ <install-dir>/opt/edg/bin/run_test.sh -x -t Testbed


Global Site Testing:

This Site Testing suite was designed to be as compact and as fast as possible.
It is based on the MainScript facility provided into the edg-tests module: a specific README file is available.
Details are also available in Barcelona slides (ppt, pdf), and a more recent description is available in a talk for the LCG Grid Deployment group (ppt, pdf).
Several separate tests will run in turn to check different components:

Prerequisite: Create a user proxy:

$ grid-proxy-init -valid 48:00

Using the MainScript to launch the test suite:

$ <install-dir>/opt/edg/bin/MainScript --TGroup=SiteTesting [--printDetails]
....
HTML in: /tmp/<user-name>/<date-time>/index.html
TarBall in: <UI-hostname> /tmp/<user-name>/<date-time>/tarex.tgz
Watch the double-dash (--) for options in the MainScript command line.  

An example of MainScript output in verbose mode (option --printDetails) is provided here.
An HTML formatted summary is built, and a result tarball is created for each execution.
To display this summary (file names are provided through the HTML and TarBall tags just above):

- either directly on the same machine
	$ netscape /tmp/<user-name>/<date-time>/index.html
- or after transferring the tarball
	$ cd <untar-dir>
$ sftp <UI-hostname>
> get /tmp/<user-name>/<date-time>/tarex.tgz
> quit
$ tar xzf tarex.tgz
$ netscape <untar-dir>/<date-time>/index.html

An example of such an HTML summary is accessible here. A more recent one is also available.

Please send us the result log file & tarball produced during your own testing.


TSTG Package Installation - from CVS sources:

The TSTG modules can be retrieved from the EDG CVS repository and installed as explained below.
This will allow you to modify the code if required.
In both types of install, module CVS tags have to be supplied: please refer to this tags.txt file for the existing tags
(the PRO version is recommended).

Step-by-step procedure (please check out this CVS-help for setting up your CVS access and identification):

A bash script "doing all the job for you" is also available from the grid deployment web page:

$ wget http://grid-deployment.web.cern.ch/grid-deployment/tstg/tars/install_packages
$ install_packages <edg_tests-tag> <edg_site_certification-tag>

edg-site-certification module: 

The edg-site-certification module provides functionality tests that may be launched from a UI using directly the edg-testbed-test utility: /opt/bin/edg-testbed-test (native install) or <install_dir>/opt/bin/edg-testbed-test if you have achieved a tarball or source install of the test modules.

GlobusGatekeeper: check that a Globus Gatekeeper (or a list of nodes running Gatekeepers) is correctly functionning

edg-testbed-test BaseTest::GlobusGatekeeper <host>
edg-testbed-test BaseTest::GlobusGatekeeper <host1> <host2> ...
edg-testbed-test BaseTest::GlobusGatekeeper <resource-value> ( e.q <host>/<jobmanager-contact-string>)
GridFTP:  check that basic transfers and third-party transfers work over a list of GridFTP servers (at least 2 of them)

edg-testbed-test BaseTest::GridFTP <host1> <host2> ...
EDGLifecycle: 
Single HelloWorld: 

edg-testbed-test BaseTest::EDGLifecycle::HelloWorld

HelloScript jobs: 5 jobs per stream in 2 streams

edg-testbed-test BaseTest::EDGLifecycle::HelloScript -njobs=5 -nstreams=2

Checksum jobs: 5 jobs per stream in 2 streams - checksum over a 1Mb transferred file

edg-testbed-test BaseTest::EDGLifecycle::Checksum -njobs=5 -nstreams=2 -sizemb=1

BrokerInfo Test:

edg-testbed-test BaseTest::EDGLifecycle::BrokerInfo
Information system: 
CheckGRIS: Look up at available GRIS and query each one to ensure it is alive

edg-testbed-test BaseTest::EDGLifecycle::CheckInfo::CheckGRIS -mdshost=<MDS-hostname> -mdsport=2135

CheckTree: start from a given host, descend to all GIISes and GRIses below and check that they are alive.

edg-testbed-test BaseTest::EDGLifecycle::CheckInfo::CheckTree -mdshost=<MDS-hostname> -mdsport=2135
Data Management: 
LRCMapping: check PFNs to GUID mapping

edg-testbed-test BaseTest::TestLRCMapping -i -rlshost=<RLS-hostname>

LRCAttributes: check new Attribute definition to PFN

edg-testbed-test BaseTest::TestLRCAttributes -i -rlshost=<RLS-hostname>

RMCMapping: check aliases to GUID mapping

edg-testbed-test BaseTest::TestRMCMapping -i -rlshost=<RLS-hostname>

LCG – Grid Deployment – Certification & Testing Section
Maintained by the TSTG group.
Send us suggestions and additions through:
hep-project-grid-tstg@cern.ch