Generic Installation and Configuration



Document identifier: LCG-GIS-MI
Date: 23 November 2006
Author: Guillermo Diez-Andino, Laurence Field, Oliver Keeble, Antonio Retico, Alessandro Usai, Louis Poncet
Version: v3.0.0
Abstract: gLite Generic Installation Guide

Contents

Introduction to Manual Installation and Configuration

This document is addressed to Site Administrators in charge of middleware installation and configuration.
It is a generic guide to manual installation and configuration for any supported node types.
It provides a fast method to install and configure the gLite middleware on the various node types (WN, UI, CE, SE ...) on the top of the following Linux distributions. The proposed installation and configuration method is based on the Debian apt-get tool and on a set of shell scripts built within the yaim framework [2].
The provided scripts can be used by Site Administrators with no need for in-depth knowledge of specific middleware configuration details.
three configuration files, according to provided examples.
The resulting configuration is a default site configuration. Local customizations and tuning of the middleware, if needed, can then be done manually1.

New versions of this document will be distributed synchronously with the middleware releases and they will contain the current ``state-of-art'' of the installation and configuration procedures.
A dual document with the upgrade procedures to manually update the configuration of the nodes from the previous LCG/gLite version to the current one is also part of the release.

Since the release LCG-2_3_0, the manual installation and configuration of nodes is supported by a set of scripts.
Nevertheless, the automatic configuration for some particular node types has been intentionally left not covered. The ``supported'' node types are:

For the node types above listed both installation and configuration scripts are provided.

If you want a deeper introuduction to the installation and configuration

of the specific systems (DPM, LFC, dCache...), you should visit page http://goc.grid.sinica.edu.tw/gocwiki/SystemsConfigGuides

OS Installation

The current version of the gLite Middleware runs on Scientific Linux 3 (SL3).
We strongly recommend all the LCG production sites to switch as soon as possible at least their service nodes to the SL3 operating system.
In that order we give here a link to the web page with all the needed information is the following:

http://www.scientificlinux.org

The site where the sources, and the images (iso) to create the CDs can be found is

ftp://ftp.scientificlinux.org/linux/scientific/30x/iso/

Most middleware testing has been carried out on CERN Scientific Linux 3 (SLC3)

http://linuxsoft.cern.ch/

but should run on any binary compatible distribution.


Java Installation

You should install java sdk 1.4.2 on your system before installing the middleware. Download it from SUN java web site (1.4.2 is required - http://java.sun.com/j2se/1.4.2/download.html). You should absolutely install the J2SDK 1.4.2 rpm package (if you do not install it in RPM format you'll not be able to install the middleware), on the sun java web page follow the link RPM in a self extracting file. Then follow instructions provided by SUN.
Set in your site-info.def (YAIM configuration file) the variable JAVA_LOCATION to your java installation directory.


Node Synchronization

A general requirement for the gLite nodes is that they are synchronized. This requirement may be fulfilled in several ways. If your nodes run under AFS most likely they are already synchronized. Otherwise, you can use the NTP protocol with a time server.

Instructions and examples for a NTP client configuration are provided in this section. If you are not planning to use a time server on your machine you can just skip it.

NTP Software Installation

Use the latest ntp version available for your system. If you are using APT, an apt-get install ntp will do the work.

NTP Configuration

Configuration Tool: YAIM

Note on yaim and gLite nodes

Where yaim is configuring a gLite node type, it populates the XML files and runs the gLite config scripts. Please note that any modifications you make to the XML files, to parameters not managed by yaim, should be {bf preserved. Parameters managed by yaim will be clearly marked in the XML after it has been run. The intention is that yaim offers a simple interface if prefered, but the ability to use the more powerful native machanism is retained.

Please use yaim to configure pool accounts. Yaim allows non-contiguous ranges of uids which some sites require and is therefore the default user configuration mechanism.

Installing yaim

From now on we will refer to the node to be installed as the target node
In order to work with the yaim installation and configuration tool yaim must be installed on the target node.
In order to download yaim:

Site Configuration File

All the configuration values relevant to sites have to be configured in a site configuration file using key-value pairs.
The site configuration file is shared among all the different node types. So we suggest to edit it once and keep it in a safe place in order not to have to edit it again for each installation.
Modifications possibly occurring in the specification of the site configuration file will be published with this document.
An up-to-date example of site configuration file is anyway provided in the file /opt/glite/yaim/examples/site-info.def


Specification of the Site Configuration File

The general syntax of the file is a sequence of bash-like assignments of variables ($<$variable$>$=$<$value$>$).
"Comment" characters "#" can be used within the file.

WARNING: The Site Configuration File is sourced by the configuration scripts. Therefore there must be no spaces around the equal sign.

Example of wrong configuration:

SITE_NAME = my-site
Example of correct configuration:
SITE_NAME=my-site
A good syntax test for your Site Configuration file (e.g. my-site-info.def) is to try and source it manually, running the command
> source my-site-info.def
and checking that no error messages are produced.

The complete specification of the configurable variables follows. Remember that in the 'examples' directory of yaim there is a commented example site-info.def file which can give further explanation about these variables.

If you need to configure a limited set of nodes, you will not need to supply values for all these variables.

APEL_DB_PASSWORD :
database password for apel.
BATCH_BIN_DIR :
The path of the lrms commands, eg /usr/pbs/bin.
BATCH_LOG_DIR :
Your batch system log directory.
BATCH_VERSION :
The version of the Local Resource Managment System, eg OpenPBS_2.3.
BDII_FCR :
Set the URL of the Freedom of Choice for Rescources URL.
BDII_HOST :
BDII Hostname.
BDII_HTTP_URL :
URL pointing to the BDII configuration file.
BDII_REGIONS :
List of node types publishing information on the bdii. For each item listed in the BDII_REGIONS variable you need to create a set of new variables as follows:
BDII_$<$REGION$>$_URL :
URL of the information producer (e.g.: BDII_CE_URL="URL of the CE information producer", BDII_SE_URL="URL of the SE information producer".
CA_REPOSITORY :
APT repository with Certification Authorities (use the one in the example).
CE_BATCH_SYS :
Implementation of site batch system. Available values are ``torque'', ``lsf'', ``pbs'', ``condor'' etc.
CE_CPU_MODEL :
Model of the CPU used by the WN (WN specification). This parameter is a string whose domain is not defined yet in the GLUE Schema. The value used for Pentium III is "PIII".
CE_CPU_SPEED :
Clock frequency in Mhz (WN specification).
CE_CPU_VENDOR :
Vendor of the CPU. used by the WN (WN specification). This parameter is a string whose domain is not defined yet in the GLUE Schema. The value used for Intel is ``intel''.
CE_HOST :
Computing Element Hostname.
CE_INBOUNDIP :
TRUE if inbound connectivity is enabled at your site, FALSE otherwise (WN specification).
CE_MINPHYSMEM :
RAM size in kblocks (WN specification).
CE_MINVIRTMEM :
Virtual Memory size in kblocks (WN specification).
CE_OS :
Operating System name (WN specification) - see http://goc.grid.sinica.edu.tw/gocwiki/How_to_publish_the_OS_name.
CE_OS_RELEASE :
Operating System release (WN specification) - see http://goc.grid.sinica.edu.tw/gocwiki/How_to_publish_the_OS_name.
CE_OS_VERSION :
Operating System Version (WN specification) - see http://goc.grid.sinica.edu.tw/gocwiki/How_to_publish_the_OS_name.
CE_OUTBOUNDIP :
TRUE if outbound connectivity is enabled at your site, FALSE otherwise (WN specification).
CE_RUNTIMEENV :
List of software tags supported by the site. The list can include VO-specific software tags. In order to assure backward compatibility it should include the entry 'LCG-2', the current middleware version and the list of previous middleware tags.
CE_SF00 :
Performance index of your fabric in SpecFloat 2000 (WN specification). For some examples of Spec values see http://www.specbench.org/osg/cpu2000/results/cint2000.html.
CE_SI00 :
Performance index of your fabric in SpecInt 2000 (WN specification). For some examples of Spec values see http://www.specbench.org/osg/cpu2000/results/cint2000.html.
CE_SMPSIZE :
Number of cpus in an SMP box (WN specification).
CLASSIC_HOST :
The name of your SE_classic host.
CLASSIC_STORAGE_DIR :
The root storage directory on CLASSIC_HOST.
CRON_DIR :
Yaim writes all cron jobs to this directory. Change it if you want to turn off Yaim's management of cron.
DCACHE_ADMIN :
Host name of the server node which manages the pool of nodes.
DCACHE_POOLS :
List of pool nodes managed by the DCACHE_ADMIN server node.
DCACHE_PORT_RANGE :
DCACHE Port Range. This variable is optional and the default value is "20000,25000" .
DPMDATA :
Directory where the data is stored (absolute path, e.g./storage).
DPMFSIZE :
The maximum file size managed (e.g.200M).
DPMMGR :
db user account for the DPM.
DPMPOOL :
Name of the Pool (per default Permanent).
DPMPOOL_NODES :
Space separated list of DPM pool hostname:/path entries".
DPMUSER_PWD :
Password of the db user account.
DPM_DB_HOST :
Set this if your DPM server uses a db on a separate machine. Defaults to localhost.
DPM_HOST :
Host name of the DPM host, used also as a default DPM for the lcg-stdout-mon .
EDG_WL_SCRATCH :
Optional scratch directory for jobs.
EDG_WL_SCRATCH :
Optional scratch directory for jobs.
FTS_HOST :
hostname of your FTS server - use this only if building an FTS server.
FTS_SERVER_URL :
URL of the File Transfer Service server.
FUNCTIONS_DIR :
The directory where yaim will find its functions.
GLOBUS_TCP_PORT_RANGE :
Port range for Globus IO.
GRIDICE_SERVER_HOST :
GridIce server host name (usually run on the MON node).
GRIDMAP_AUTH :
List of ldap servers in edg-mkgridmap.conf which authenticate users.
GRID_TRUSTED_BROKERS :
List of the DNs of the Resource Brokers host certificates which are trusted by the Proxy node (ex: /O=Grid/O=CERN/OU=cern.ch/CN=host/testbed013.cern.ch).
GROUPS_CONF :
Path to the groups.conf file which contains information on mapping VOMS groups and roles to local groups. An example of this configuration file is given in /opt/lcg/yaim/examples/groups.conf.
GSSKLOG :
yes or no, indicating whether the site provides an AFS authentication server which maps gsi credentials into Kerberos tokens .
GSSKLOG_SERVER :
If GSSKLOG is yes, the name of the AFS authentication server host.
INSTALL_ROOT :
Installation root - change if using the re-locatable distribution.
JAVA_LOCATION :
Path to Java VM installation. It can be used in order to run a different version of java installed locally.
JOB_MANAGER :
The name of the job manager used by the gatekeeper.
LCG_REPOSITORY :
APT repository with LCG middleware (use the one in the example).
LFC_CENTRAL :
A list of VOs for which the LFC should be configured as a central catalogue.
LFC_DB_PASSWORD :
db password for LFC user.
LFC_HOST :
Set this if you are building an LFC_HOST, not if you're just using clients.
LFC_LOCAL :
Normally the LFC will support all VOs in the VOS variable. If you want to limit this list, add the ones you need to LFC_LOCAL. For each item listed in the VOS variable you need to create a set of new variables as follows:
VO_$<$VO-NAME$>$_QUEUES :
The queues that the VO can use on the CE.
VO_$<$VO-NAME$>$_SE :
Default SE used by the VO. WARNING: VO-NAME must be in capital cases.
VO_$<$VO-NAME$>$_SGM :
ldap directory with VO software managers list. WARNING: VO-NAME must be in capital cases.
VO_$<$VO-NAME$>$_STORAGE_DIR :
Path to the storage area for the VO on an SE_classic. WARNING: VO-NAME must be in capital cases.
VO_$<$VO-NAME$>$_SW_DIR :
Area on the WN for the installation of the experiment software. If on the WNs a predefined shared area has been mounted where VO managers can pre-install software, then these variable should point to this area. If instead there is not a shared area and each job must install the software, then this variables should contain a dot ( . ).Anyway the mounting of shared areas, as well as the local installation of VO software is not managed by yaim and should be handled locally by Site Administrators. WARNING: VO-NAME must be in capital cases.
VO_$<$VO-NAME$>$_USERS :
ldap directory with VO users list. WARNING: VO-NAME must be in capital cases.
VO_$<$VO-NAME$>$_VOMSES :
List of entries for the vomses files for this VO. Multiple values can be given if enclosed in single quotes.
VO_$<$VO-NAME$>$_VOMS_EXTRA_MAPS :
Add any further arbitrary maps you need in edg-mkgridmap.conf - see example site-info.def in yaim .
VO_$<$VO-NAME$>$_VOMS_POOL_PATH :
If necessary, append this to the VOMS_SERVER URL for the pool account list .
VO_$<$VO-NAME$>$_VOMS_SERVERS :
A list of VOMS servers for the VO.
MON_HOST :
MON Box Hostname.
MYSQL_PASSWORD :
mysql password for the accounting info collector.
MY_DOMAIN :
site's domain name.
OUTPUT_STORAGE :
Default Output directory for the jobs.
PX_HOST :
PX hostname.
QUEUES :
The name of the queues for the CE. These are by default set as the VO names.
RB_HOST :
Resource Broker Hostname.
RB_RLS :
The RB now uses the DLI by default; set VOs here which should use RLS.
REG_HOST :
RGMA Registry hostname.
REPOSITORY_TYPE :
apt or yum.
RESET_DCACHE_CONFIGURATION :
Set this to yes if you want YAIM to configure dCache for you - if unset (or 'no') yaim will only configure the grid front-end to dCache.
RFIO_PORT_RANGE :
Optional variable for the port range with default value "20000,25000".
SE_ARCH :
=defaults to multidisk - "disk, tape, multidisk, other" - populates GlueSEArchitecture.
SE_LIST :
A list of hostnames of the SEs available at your site.
SITE_EMAIL :
The e-mail address as published by the information system.
SITE_LAT :
Site latitude.
SITE_LOC :
"City, Country".
SITE_LONG :
Site longitude.
SITE_NAME :
Your GIIS.
SITE_SUPPORT_SITE :
Support entry point ; Unique Id for the site in the GOC DB and information system.
SITE_TIER :
Site tier.
SITE_WEB :
Site site.
TORQUE_SERVER :
Set this if your torque server is on a different host from the CE. It is ingored for other batch systems.
USERS_CONF :
Path to the file containing a list of Linux users (pool accounts) to be created. This file should be created by the Site Administrator, which contains a plain list of the users and IDs. An example of this configuration file is given in /opt/lcg/yaim/examples/users.conf.
VOBOX_HOST :
VOBOX hostname.
VOBOX_PORT :
The port the VOBOX gsisshd listens on.
VOS :
List of supported VOs.
VO_SW_DIR :
Directory for installation of experiment software.
WMS_HOST :
Hostname of the gLite WMS/LB server.
WN_LIST :
Path to the list of Worker Nodes. The list of Worker Nodes is a file to be created by the Site Administrator, which contains a plain list of the batch nodes. An example of this configuration file is given in /opt/lcg/yaim/examples/wn-list.conf.
YAIM_VERSION :
The version of yaim for which this config file is valid.

Configuring VO support

VOs can advertise their parameters via the YAIM VO Configurator;

https://lcg-sft.cern.ch/yaimtool/yaimtool.py

Site administrators can use this utility to maintain a list of the VOs their site supports and to automatically generate the appropriate YAIM fragment.


RPM Installation Tool:APT-GET

Please before you proceed further MAKE SURE that Java is installed in your system (see 3.).

Please note that for the dependencies of the middleware to be met, you'll have to make sure that apt can find and download your OS rpms. This typically means you'll have to install an rpm called 'apt-sourceslist', or else create an appropriate file in your /etc/apt/sources.list.d directory.

Notes on using RHEL3 compatible distributions other than CERN Scientific Linux

If you are not using SLC3 but another OS binary compatible distribution is highly recommended that you configure apt-get in order to give priority, during the installation, to packages listed within your distribution.

In order to have all the known dependencies possibly solved by apt-get you should have at least the following lists in your /etc/apt/sources.list.d/:

The first two are distributed by the 'apt-sourceslist' rpm, the third one is your local one.

Since the deployment team is based at CERN and it uses the local installation, it is still possible that with this bare configuration, some dependencies, though dealt with, cannot be solved because the binary compatible distribution you use does not provide the entire set of packages which CERN SL3 does.

If you prefer not to handle these issues manually you could add in the /etc/apt/sources.list.d/ another list (e.g. cern.list)

### List of available apt repositories available from linuxsoft.cern.ch
### suitable for your system.
###
### See http://cern.ch/linux/updates/ for a list of other repositories and mirrors.
### 09.06.2004
###

# THE default
rpm http://linuxsoft.cern.ch  cern/slc30X/i386/apt  os updates extras
rpm-src http://linuxsoft.cern.ch  cern/slc30X/i386/apt  os updates extras

Then you have to configure your apt-get preferences in order to give priority to your Os and not to CERN SLC3.

A /etc/apt/preferences file like the following one will give priority to your Os in any case except when the package that you need is not present in your-os repository :

Package: *
Pin: release o=your-os.your-domain.org
Pin-Priority: 980
Package: *
Pin: release o=linux.cern.ch
Pin-Priority: 970

If you are not using apt to install, you can pull the packages directly from SLC3's repository using wget. The address is http://linuxsoft.cern.ch/cern/slc305/i386/apt/.


Middleware Installation

In order to install the node with the desired middleware packages run the command

> /opt/glite/yaim/scripts/install_node <site-configuration-file> <meta-package> [ <meta-package> ... ]

The complete list of the available meta-packages available with this release is provided in 8.1.(SL3)

For example, in order to install a CE with Torque, after the configuration of the site-info.def file is done, you have to run:

> /opt/glite/yaim/scripts/install_node site-info.def lcg-CE_torque

WARNING: There is a known installation conflict between the 'torque-clients' rpm and the 'postfix' mail server (Savannah. bug #5509).
In order to workaround the problem you can either uninstall postfix or remove the file /usr/share/man/man8/qmgr.8.gz from the target node.

The ``bare-middleware'' versions of the WN and CE meta-packages are provided in case you have an existing LRMS;

> /opt/glite/yaim/scripts/install_node site-info.def lcg-CE

You can install multiple node types on one machine

> /opt/glite/yaim/scripts/install_node site-info.def <meta-package> <meta-package> ...


meta-packages-SL3

In the following table the list of SL3 meta-packages is provided.

Table 1: meta-packages available for SL3
Node Type meta-package Name meta-package Description
gLite WMS and LB glite-WMSLB Combined WMS LB node
glite CE glite-CE The gLite Computing Element
FTS glite-FTS gLite File Transfer Server
FTA glite-FTA gLite File Transfer Agent
BDII glite-BDII BDII
LCG Computing Element (middleware only) lcg-CE It does not include any LRMS
LCG Computing Element (with Torque) lcg-CE_torque It includes the 'Torque' LRMS
LCG File Catalog (mysql) glite-LFC_mysql LCG File Catalog
LCG File Catalog (oracle) glite-LFC_oracle LCG File Catalog
MON-Box glite-MON RGMA-based monitoring system collector server
MON-Box glite-MON_e2emonit MON plus e2emonit
Proxy glite-PX Proxy Server
Resource Broker lcg-RB Resource Broker
Classic Storage Element glite-SE_classic Storage Element on local disk
dCache Storage Element glite-SE_dcache Storage Element interfaced to dCache without pnfs dependency
dCache Storage Element glite-SE_dcache_gdbm Storage Element interfaced to dCache with dependency on pnfs (gdbm)
DPM Storage Element (mysql) glite-SE_dpm_mysql Storage Element with SRM interface
DPM Storage Element (Oracle) glite-SE_dpm_oracle Storage Element with SRM interface
DPM disk glite-SE_dpm_disk Disk server for a DPM SE
Dependencies for the re-locatable distribution glite-TAR This package can be used to satisfy the dependencies of the relocatable distro
User Interface glite-UI User Interface
VO agent box glite-VOBOX Agents and Daemons ©
Worker Node (middleware only) glite-WN It does not include any LRMS


Certification Authorities

The installation of the up-to-date version of the Certification Authorities (CA) is automatically done by the Middleware Installation described in 8.
Anyway, as the list and structure of Certification Authorities (CA) accepted by the LCG project can change independently of the middleware releases, the rpm list related to the CAs certificates and URLs has been decoupled from the standard gLite/LCG release procedure. You should consult the page

http://grid-deployment.web.cern.ch/grid-deployment/lcg2CAlist.html

in order to ascertain what the version number of the latest set of CA rpms is. In order to upgrade the CA list of your node to the latest version, you can simply run on the node the command:
> apt-get update && apt-get -y install lcg-CA

In order to keep the CA configuration up-to-date on your node we strongly recommend Site Administrators to program a periodic upgrade procedure of the CA on the installed node (e.g. running the above command via a daily cron job).


Host Certificates

All nodes except UI, WN and BDII require the host certificate/key files before you start their installation.
Contact your national Certification Authority (CA) to understand how to obtain a host certificate if you do not have one already.
Instruction to obtain a CA list can be found in
http://grid-deployment.web.cern.ch/grid-deployment/lcg2CAlist.html

From the CA list so obtained you should choose a CA close to you.

Once you have obtained a valid certificate, i.e. a file

make sure to place the two files in the target node into the directory and check the access right hostkey.pem only readable by root and the certificate readable by everybody.

/etc/grid-security

Middleware Configuration

The general procedure to configure the middleware packages that have been installed on the node via the procedure described in 8., is to run the command:

> /opt/glite/yaim/scripts/configure_node <site-configuration-file> <node-type> [ <node-type> ... ]
For example, in order to configure the WN with Torque you had installed before, after the configuration of the site-info.def file is done, you have to run:
> /opt/glite/yaim/scripts/configure_node site-info.def WN_torque

In the following paragraph a reference to all the available configuration scripts is given.


Available Node Types

In this paragraph a reference to all the available configuration scripts is given.
For those items in the list tagged with an asterisk (*), there are some particularities in the configuration procedure or extra configuration details to be considered, which are described in a following dedicated section.
For all the unmarked node types, the general configuration procedure is the one above described.



Table 2: Available Node Types
Node Type Node Type Node Description
gLite WMS and LB WMSLB Combined WMS LB node
glite CE gliteCE The gLite Computing Element
FTS FTS gLite File Transfer Server
FTA FTA gLite File Transfer Agent
BDII BDII A top level BDII
Computing Element (middleware only) CE It does not configure any LRMS
Computing Element (with Torque) * CE_torque It configures also the 'Torque' LRMS client and server (see 12.6. for details)
LCG File Catalog server * LFC_mysql Set up a mysql based LFC server
MON-Box MON RGMA-based monitoring system collector server
e2emonit E2EMONIT RGMA-based monitoring system collector server
Proxy PX Proxy Server
Resource Broker RB Resource Broker
Classic Storage Element SE_classic Storage Element on local disk
Disk Pool Manager (mysql) * SE_dpm_mysql Storage Element with SRM interface and mysql backend
Disk Pool Manager disk * SE_dpm_disk Disk server for SE_dpm
dCache Storage Element SE_dcache Storage Element interfaced with dCache
Re-locatable distribution * TAR_UI or TAR_WN It can be used to set up a Worker Node or a UI (see 12.9. for details)
User Interface UI User Interface
VO agent box VOBOX Machine to run VO agents
Worker Node (middleware only) WN It does not configure any LRMS
Worker Node (with Torque client) WN_torque It configures also the 'Torque' LRMS client


Installing multiple node types on one machine

You can use yaim to install more than one node type on a single machine. In this case, you should install all the relevant software first, and then run the configure script. For example, to install a combined RB and BDII, you should do the following;

> /opt/glite/yaim/scripts/install_node site-info.def RB BDII
> /opt/glite/yaim/scripts/configure_node site-info.def RB BDII

All node-types must be given as arguments to the same invocation of configure_node - do not run this command once for each node type. Note that combinations known not to work are the CE/RB, RB/SE, CE/BDII.

Node-specific installation advice

In this section we list configuration steps actually needed to complete the configuration of the desired node but not supported by the automatic configuration scripts.
If a given node does not appear in that section it means that its configuration is complete

gLite WMS/LB

To install the glite WMS + glite LB (recommended deployment scenario)
install_node site-info.def glite-WMSLB
configure_node site-info.def WMSLB

gLite CE

Normal gLite CE configuration has no site BDII;

install_node site-info.def glite-CE
configure_node site-info.def gliteCE

site BDII on the gLite CE

If you want your gliteCE to run the site BDII;

configure_node site-info.def gliteCE BDII_site

Due to a yaim bug (fixed in later versions) you'll have to add

BDII_site_FUNCTIONS=$BDII_FUNCTIONS

in yaim's scripts/node-info.def (after the BDII_FUNCTIONS definition) for this to work.

BDII_site is just a mechanism to allow a gLiteCE to run a site level BDII, it is not a standard configuration target or node type (there is no associated meta-rpm).

Place a file containing the following in $LCG_LOCATION/var/gip/ldif

dn: GlueSiteUniqueID=<YourSiteName>,mds-vo-name=local,o=grid
objectClass: GlueTop
objectClass: GlueSite
objectClass: GlueKey
objectClass: GlueSchemaVersion
GlueSiteUniqueID: <YourSiteName>
GlueSiteName: <YourSiteame>
GlueSiteDescription: LCG Site
GlueSiteUserSupportContact: mailto: <YourserSupportContactMail>
GlueSiteSysAdminContact: mailto: <YourSysAdminContactMail>
GlueSiteSecurityContact: mailto: <YourSecurityContactMail>
GlueSiteLocation: <City>, <Country>
GlueSiteLatitude: <SiteLatitude>
GlueSiteLongitude: <SiteLongitude>
GlueSiteWeb: <YourSiteWeb>
GlueSiteSponsor: none
GlueSiteOtherInfo: TIER 2
GlueSiteOtherInfo: my-bigger-site.domain
GlueForeignKey: GlueSiteUniqueID=<YourSiteName>
GlueSchemaVersionMajor: 1
GlueSchemaVersionMinor: 2

lcg-info-dynamic-scheduler

The glite-CE configuration configures also software and scheduler GIP plugins. Due to the bug in the /opt/lcg/libexec/lcg-info-dynamic-scheduler file the following command must be run in order to get a correct functionality:

# sed -i '{s/jobmanager/blah/}' /opt/lcg/libexec/lcg-info-dynamic-scheduler

Batch systems and the gLite CE

If you are installing your batch system server on the same node as the CE, and you want to use yaim or gLite to configure it, please choose one or the other and stick to it. If you use yaim and then make modifications via the gLite system, any rerun of yaim will reset the configuration. The same advice applies to management of WNs. If yaim fulfils your needs, this is the recommended route.

glite-CE + Torque server

install_node site-info.def glite-CE glite-torque-server-config
configure_node site-info.def gliteCE TORQUE_server

TORQUE_server is a configuration target provided to help configure torque with the gliteCE or on a separate machine. There is no directly associated meta-rpm, but please use glite-torque-server-config to combine with the gliteCE (as illustrated above).

Separate Torque server

Note that the log-parser daemon must be started on whichever node is running the batch system. If your CE node is also the batch system head node, you have to run the log-parser here.

If you are running two CEs (typically LCG and gLite versions) please take care to ensure no collisions of pool account mapping. This is typically achieved either by allocating separate pool account ranges to each CE or by allowing them to share a gridmapdir.

LSF

Information on running with LSF can be found here https://uimon.cern.ch/twiki/bin/view/LCG/LSFCeExtraSteps

SGE

Information on Sun Grid Engine can be found here http://goc.grid.sinica.edu.tw/gocwiki/How_to_run_an_SGE_farm_on_LCG

MON and E2EMONIT

You can add E2EMONIT to your MON box like this

install_node site-info.def glite-MON_e2emonit
configure_node site-info.def MON E2EMONIT

WN

To install the glite-WN + Torque client

install_node site-info.def glite-WN glite-torque-client-config
configure_node site-info.def WN_torque


LCG CE_torque

WARNING: in the CE configuration context (and also in the 'torque' LRMS one), a file with a a list of managed nodes needs to be compiled. An example of this configuration file is given in /opt/glite/yaim/examples/wn-list.conf
Then the file path needs to be pointed by the variable WN_LIST in the Site Configuration File (see 6.1.).

The Maui scheduler configuration provided with the script is currently very basic.
More advanced configuration examples, to be implemented manually by Site Administrators can be found in [6]

FTS

There is still a manual step required in configuring FTS

https://uimon.cern.ch/twiki/bin/view/LCG/FtsServerInstall15

At the present time, the FTS requires a different proxy server to that used by the broker. Please ensure this restriction is respected in the site-info.def file you use to configure the File Transfer Server.

Please see the FTS install guides for more information

https://uimon.cern.ch/twiki/bin/view/LCG/FtsRelease15

https://uimon.cern.ch/twiki/bin/view/LCG/FtsServerInstall15

dCache

Yaim does not yet support d-Cache with a postgresql based pnfs. To accommodate sites who have already upgraded to this version of pnfs, we now have two types of d-Cache SE.

glite-SE_dcache

This has no dependency on pnfs at all, so upgrades of either type (postgresql or gdbm) should work at the rpm level.

glite-SE_dcache_gdbm

This has a dependency on pnfs (ie the gdbm version) and is necessary for a new install. Please note however that pnfs_postgresql is the preferred implementation and migration is non trivial.


The relocatable distribution

Introduction

We are now supplying a tarred distribution of the middleware which can be used to install a UI or a WN. It can be used on Debian as well as SL3. You can untar the distribution somewhere on a local disk, or replicate it across a number of nodes via a network share. You can also use this distribution to install a UI without root privileges - there is a quick guide to doing this here [5].

Once you have the middleware directory available, you must edit the site-info.def file as usual, putting the location of the middleware into the variable INSTALL_ROOT.

If you are sharing the distribution to a number of nodes, commonly WNs, then they should all mount the tree at INSTALL_ROOT. You should configure the middleware on one node (remember you'll need to mount with appropriate privileges) and then it should work for all the others if you set up your batch system and the CA certificates in the usual way. If you'd rather have the CAs on your share, the yaim function install_certs_userland may be of interest. You may want to mount your share ro after the configuration has been done.

Dependencies

The middleware in the relocatable distribution has certain dependencies.

We've made this software available as a second tar file which you can download and untar under $INSTALL_ROOT. This means that if you untarred the main distribution under /opt/LCG, you must untar the supplementary files in the same place. Please note that in earlier distributions the deps were untarred elsewhere.

If you have administrative access to the nodes, you could alternatively use the TAR dependencies rpm.

> /opt/glite/yaim/scripts/install_node site-info.def glite-TAR

For Debian, here is a list of packages which are required for the tarball to work

perl-modules python2.2 libexpat1 libx11-6 libglib2.0-0 libldap2 libstdc++2.10-glibc2.2 tcl8.3-dev 
libxml2 termcap-compat libssl0.9.7 tcsh rpm rsync cpp gawk openssl wget

To configure a UI or WN

Run the configure_node script, adding the type of node as an argument;

> /opt/glite/yaim/scripts/configure_node site-info.def [ TAR_WN | TAR_UI ]

Note that the script will not configure any LRMS. If you're configuring torque for the first time, you may find the config_users and config_torque_client yaim functions useful. These can be invoked like this

${INSTALL_ROOT}/glite/yaim/scripts/run_function site-info.def config_users
${INSTALL_ROOT}/glite/yaim/scripts/run_function site-info.def config_torque_client

Installing a UI as a non-root user

You can find a quick guide to this here [5].

If you don't have root access, you can use the supplementary tarball mentioned above to ensure that the dependencies of the middleware are satisfied. The middleware requires java (see 3.), which you can install in your home directory if it's not already available. Please make sure you set the JAVA_LOCATION variable in your site-info.def. You'll probably want to alter the OUTPUT_STORAGE variable there too, as it's set to /tmp/jobOutput by default and it may be better pointing at your home directory somewhere.

Once the software is all unpacked, you should run

> $INSTALL_ROOT/glite/yaim/scripts/configure_node site-info.def TAR_UI
to configure it.

Finally, you'll have to set up some way of sourcing the environment necessary to run the grid software. A script will be available under $INSTALL_ROOT/etc/profile.d for this purpose. Source grid_env.sh or grid_env.csh depending upon your choice of shell.

Installing a UI this way puts all the CA certificates under $INSTALL_ROOT/etc/grid-security and adds a user cron job to download the crls. However, please note that you'll need to keep the CA certificates up to date yourself. You can do this by running

> /opt/glite/yaim/scripts/run_function site-info.def install_certs_userland

Further information

In [3] there is more information on using this form of the distribution. You should check this reference if you'd like to customise the relocatable distribution.

This distribution is used at CERN to make its lxplus system available as a UI. You can take a look at the docs for this too [4].

Getting the software

You can download the latest gliteUI_WN-3.x.y-z.tar.gz and gliteUI_WN-3.x.y-z-userdeps.tar.gz tar files from

http://grid-deployment.web.cern.ch/grid-deployment/download/relocatable/

VOBOX

Site admins must ensure that the experiment software installation area is accessible (i.e. mounted) from the VOBOX.
In the VOBOX installation it is crucial to have the $MYPROXY_SERVER env variable (PX_HOST in the yaim site-info.def) set to the CERN myproxy server (myproxy.cern.ch). Even if you have a private myproxy server in your site, configure the VOBOX to point to the CERN one.
The site administrator must communicate the name of the VOBOX to the myproxy.cern.ch service administrator (email both hep-project-grid-cern-testbed-managers@cern.ch and support-eis@cern.ch ) so that it is included in the list of authorized renewers. If this is not done, the renewal agent of the VOBOX will not work.

Firewalls

No automatic firewall configuration is provided by this version of the configuration scripts.
If your nodes are behind a firewall, you will have to ask your network manager to open a few "holes" to allow external access to some service nodes.
A complete map of which port has to be accessible for each service node is maintined in CVS; http://jra1mw.cvs.cern.ch:8180/cgi-bin/jra1mw.cgi/org.glite.site-info.ports/doc/?only_with_tag=HEAD

Support

Your contact point for support is your ROC;

http://egee-sa1.web.cern.ch/egee-sa1/ROC-support.htm

Bibliography

1
G. Diez-Andino, K. Oliver, A. Retico, and A. Usai.
Lcg configuration reference, 2004.
http://www.cern.ch/grid-deployment/gis/lcg-GCR/index.html.

2
L. Field and L. Poncet.
The new manual install, 2004.
http://agenda.cern.ch/askArchive.php?base=agenda&categ=a044377&id=a044377s11t3/transparencies
Presentation given at the LCG Workshop on Operational Issues, Cern, Nov 2.

3
O. Keeble.
Lcg 2 tar distribution, 2004.
http://grid-deployment.web.cern.ch/grid-deployment/documentation/Tar-Dist-Use/.

4
O. Keeble.
Using lxplus as a ui, 2004.
http://www.cern.ch/grid-deployment/documentation/UI-lxplus/.

5
O. Keeble.
How to quickly set up a ui without root privileges, 2005.
http://grid-deployment.web.cern.ch/grid-deployment/documentation/quickUI/.

6
S. Lemaitre and S. Traylen.
Maui-cookbook, 2004.
http://grid-deployment.web.cern.ch/grid-deployment/documentation/Maui-Cookbook.pdf.


Change History



Table 3: Change History
version date description
v2.5.0-1 17/Jul/05 Removing Rh 7.3 support completely.
v2.3.0-2 10/Jan/05 6.1.: CA_WGET variable added in site configuration file.
v2.3.0-3 2/Feb/05 Bibliography: Link to Generic Configuration Reference changed.
" " 12.6., 6.1.: Details added on WN and users lists.
" " script ``configure_torque''. no more available: removed from the list.
v2.3.0-4 16/Feb/05 Configure apt to find your OS rpms.
v2.3.0-5 22/Feb/05 Remove apt prefs stuff, mention multiple nodes on one box.
v2.3.0-6 03/Mar/05 Better lcg-CA update advice.
v2.3.1-1 03/Mar/05 LCG-2_3_1 locations
v2.3.4-0 01/Apr/05 LCG-2_4_4 locations
v2.3.4-1 08/Apr/05 external variables section inserted
v2.3.4-2 31/May/05 4.: fix in firewall configuration
" " 11.: verbatim line fixed
v2.5.0-0 20/Jun/05 6.1.: New variables added
" " 11.1.: New nodes added (dpm)
v2.5.0-1 28/Jun/05 7.: note on apt-get preferences added
v2.6.0-1 23/Sep/05 10.: host certificates needed on the VOBOX
v2.7.0-0 11/Jan/06 6.1.: new variables GROUPS_CONF RB_RLS BATCH_LOG_DIR CLASSIC_HOST CLASSIC_STORAGE_DIR DPMPOOL_NODES SE_LIST BDII_FCR_URL VO_XXX_VOMSES added.
v2.7.0-1 10/Feb/06 11.1.: all the forbidden combinations added
v3.0.0-1 28/Apr/06 Updated for gLite 3.0


About this document ...

Generic Installation & Configuration Guide

This document was generated using the LaTeX2HTML translator Version 2002-2-1 (1.70)

Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.

The command line arguments were:
latex2html -split 0 -html_version 4.0 -no_navigation -address 'GRID deployment' LCG2-Manual-Install.drv_html

The translation was initiated by Oliver KEEBLE on 2006-11-23


Footnotes

... manually1
A technical reference of the configuration actions done by the yaim scripts, suitable to be used to learn more about configuration can be found in [1]
... installed)2
The apt tool could be already installed according to the distribution of OS in use at your site


GRID deployment