Context Navigation

Release Notes for gLite3.1 Templates

QWG Releases
gLite Updates
Upgrading from gLite 3.0
Known Problems of Releases
Main Changes
Change Log

This page contains information about each release of QWG Templates for gLite 3.1, in particular new or changes features and known problems. To know how to configure the template, refer to the page on gLite templates customization.

QWG Releases

Note : you can have a look at ongoing developments and progress of upcoming release through Roadmap button, last entries in the gLite-3.1 branch ChangeLog or full log of trunk branch.'

Date	Release	Description
16/6/2007		Creation of branch gLite-3.1
30/6/2007	gLite-3.1.0-1	WN (32 and 64-bit), DPM disk server 64-bit
7/11/2007	gLite-3.1.0-2	UI (32 and 64-bit), update 05, CA 1.17, SL 4.5
12/3/2007	gLite-3.1.0-3	update 06-16, CA 1.19, CE, MyProxy, LFC (32/64), DPM (32/64), BDII, dCachee 1.8, Xen support
23/11/2008	gLite-3.1.0-4	update 17-34, CA 1.20-25, VOBOX, MONBOX, WMS/LB, OpenVZ support, SL 5.1/2
10/03/2009	gLite-3.1.0-5	update 35-41, CA 1.26-27, NFS v4, SL 4.7
25/6/2009	gLite-3.1.0-6	update 41-47, CA 1.28-30, DPM/xroot support, Torque/MAUI based on HW configuration
28/10/2009	gLite-3.1.0-7	gLite updates 49-55, CA 1.28-30, VOMS server, Hydra server
09/08/2010	gLite-3.1.0-8	gLite updates 56-65, CA 1.31-36, CREAM CE (multiple CEs per cluster), OS errata
09/02/2011	gLite-3.1.0-9	gLite update 69, CA 1.37, OS errata

gLite Updates

QWG templates releases deliver the last gLite updates available at the time of the release. There is no equivalent between QWG release number (-n) and gLite update numbers. Sometimes one QWG templates release deliver several gLite updates. In each QWG release, there is a default associated gLite update (generally the last one).

QWG releases provide a standard mechanism for selecting the gLite update you want to deploy on a per node, per cluster or per site basis.

Content of gLite updates and associated release notes can be viewed at http://glite.web.cern.ch/glite/packages/R3.1/updates.asp.

Upgrading from gLite 3.0

QWG release gLite-3.0.1-1 supports updating existing gLite 3.0 nodes, managed by QWG releases gLite-3.0.2 series (any version, look at gLite 3.0 release notes for any restriction).

Note : there is a potential issue when upgrading from gLite 3.0 if RPM for one component has the same version in both versions of gLite. In this case, the RPM from gLite 3.0 (built for SL3) will not be replaced by the gLite 3.1 RPM (build for SL4). This may lead to unexpected behaviours or errors. One known problem is lcg_util if you are upgrading from gLite 3.0 update 31 or later to gLite 3.1. To solve the problem you need either to force manually a reinstallation of lcg_util RPM or to first downgrade the node you want to upgrade to gLite 3.1 to a gLite update before 30.

gLite version cannot be mixed inside the same SCDB cluster. Updating existing node involves :

Creating a new cluster using the following steps :
- Copy the existing cluster with svn cp
- Edit cluster.build.properties of the new cluster to reflect the use of gLite 3.1 templates intead of gLite 3.0
- Edit repository/config.tpl to use gLite 3.1 RPM repositories instead of gLite 3.0 ones.
- Review other cluster specific templates, in particular site/pro_site_cluster_info.tpl, for any change that could be necessary (none required).
Moving node profiles from one cluster to the other with svn mv command.

Note : there is no problem to have nodes part of the same grid configuration (e.g. CE and WNs) belonging to 2 different clusters. It is recommended to put site specific gLite parameters, site/pro_lcg2_config_site.tpl, in your site directory.

Known Problems of Releases

This section describes severe issues that have been discovered after a release of QWG templates. They are generally fixed in the release preview of next releases, available in gLite 3.1 branch, which is the most up-to-date version recommended for use in production.

The version mentioned in each title is the release having the problem.

gLite-3.1.0.8: WMS/ICE database must be recreated

gLite 3.1 update 65, delivered as part of QWG templates 3.1.0-8, contains a new release of WMS. The new version of ICE component (interface to CREAM CE) requires the ICE database to be deleted if it exists before restarting the WMS. Default location of ICE db is /var/glite/ice/ice_db.

gLite-3.1.0.8: New ncm-cups version requires configuration changes

New version of ncm-cups delivered with version 3.1.0.8 of QWG templates changed the printer list format from a list to a nlist. To use this new version, you need to update your configuration. The change is fairly simple: the name property of the printer entries must be removed and its former value must be used as the nlist key.

For example if you had a printer definition like:

"/software/components/cups/printers" = push(
  nlist("name", "myprinter"
        "protocol", "LPD",
        "server", "printsrv.example.org",
        "location", "Building unkwown",
        "description", "My preferred printer type",
       ),
);

you will need to change it to:

"/software/components/cups/printers" = {
  SELF['myprinter'] = nlist("protocol", "LPD",
                            "server", "printsrv.example.org",
                            "location", "Building unkwown",
                            "description", "My preferred printer type",
                           );
  SELF
};

gLite-3.1.0-8: possible problems when first installing a CREAM CE

After installing a CREAM CE, you may encounter a few errors requiring a manual intervention for cleaning them up.

If the machine certificate is not installed during machine installation, you need to copy the certificate in /etc/grid-security and run again filecopy component with the following command:
```
ncm-ncd --configure filecopy
```
After installing the certificate you may also have to run manually fetch-crl if you want the CE to be immediatly operational. Look at the command to run in cron file /etc/cron.d/fetch-crl-cron.ncm-cron.cron. After updating CRLs, you need to restart Tomcat with the following command:
```
service tomcat5 restart
```
Generally, for some unidentifed reason, MySQL database is not initialized properly at the first run of MySQL. This can be assessed looking at /var/log/ncm-cidspd. This can be fixed by running again Quattor configuration module mysql with the following command:
```
ncm-ncd --configure mysql
```

In addition, there is sometimes a chicken and egg problem with a Tomcat application, ce-monitor. If the directory ls /usr/share/tomcat5/webapps/ce-monitor/WEB-INF contains only the classes sub-directory, you need to remove this directory and restart Tomcat:

rm -Rf ls /usr/share/tomcat5/webapps/ce-monitor
service tomcat5 restart

gLite-3.1.0-8: kernel errata deployment problem caused by a SPMA bug

This problem is not related strictly to gLite templates. It may affect any attempt to deploy kernel errata with SPMA version 1.10.34 to 1.11.1.

SPMA version 1.10.34 introduces a new feature, triggered by option protectkernel which is true by default, allowing a smooth upgrade of kernels and related modules. Kernel and related modules are not uninstalled as long as they are active, even though they are no longer part of the configuration. To implement this feature, SPMA tries to guess the kernel name and variant (smp, largesmp...). Unfortunatly the regexp used for this is buggy until SPMA 1.11.2. Depending on SPMA version it affects either largesmp variant or the other variants.

Fixed SPMA version is delivered with gLite-3.1.0-8 release of QWG templates (since r4946 of QWG trunk). But depending on the SPMA version you use when you upgrade to this version of the QWG templates, you may end up in a chicken and egg problem if you tried to deploy errata with a buggy version. The problem is that because of the SPMA bug there is dependency problem leading SPMA to fail and this prevents the upgrade of SPMA itself.

The suggested workaround is:

Roll back errata on the affected machines to a version matching what is currently deployed on the machine, so that SPMA can execute successfully.
Deploy the fixed SPMA version as provided by the last version of the templates.
Redeploy the OS errata.

gLite-3.1.0-8: Account UID changes following deployment of new VO configuration functions

As explained in the main changes section, the new VO configuration function ensure the SW manager and production user will always be assigned the same UID. This may lead to a change in the account UID numbering when first deploying the new version. This main VOs affected are: biomed, compchem, dzero, esr, fusion, flast.org, planck.

ncm-accounts has difficulties to carry on these renumbering operations. It is thus necessary to remove the potentially conflicting accounts and re-run ncm-accounts. One possibility is to use a pair of scripts installed and run by ncm-filecopy. The first one will remove the accounts. The second one will recreate them and change the ownership of home directories. Both scripts must be deployed one after the other. Example can be found in QWG examples:

site/misc/fix_fqan_accounts.tpl: script to remove the potentially conflicting accounts.
site/misc/fix_fqan_dirs.tpl: script to fix home directory permissions for modified accounts after recreating them.

After running those script, it may be necessary to fix also SW area ownership for VOs whose SW manager UID changed. This has to be done manually but this is generally not a problem as the SW area resides in a shared area.

If you are running a CE without shared home directories, there are some specific constraints when updating the VO accounts:

You have to ensure that the SSH keys for the new VO accounts are properly generated, if they are used.
When running the suggested script to remove some accounts before running ncm-accounts, you need to remove the definition of users variable to avoid having 2 accounts with the same UID as this causes an error with the scp command used to copy file between the CE and the WN. Because of this, you need to ensure that you have no jobs running under the xxxhs accounts or you need to drain the corresponding queue.

gLite-3.1.0-8: New DPM/xroot migration requires to stop manually olbd

The new DPM/xroot provided in this QWG release uses the new Cluster Management Services daemon, cmsd, instead of the legacy one, olbd. Because of a chicken and egg problem, it is not possible to stop the olbd daemon before removing the startup scripts. Unfortunatly, if olbd runs, cmsd cannot start. It is thus necessary to stop it manually on all DPM nodes and then restart cmsd.

This can be done with the following commands:

ps -e|grep olbd
kill -TERM of each process (there may be 2 on the head node)
service dpm-cms restart
service dpm-manager-cmsd restart (on the head node only)

gLite-3.1.0-7: VO_GRIDMAPFILE_FQAN_FILTER added to restrict FQANs in gridmap file

It is now possible to restrict VOMS groups and roles in VO that are added to the gridmap file. This is especially useful on VO boxes where generally only a small subset of VO members are allowed to log in using gsissh. Note that this has no impact on services that use LCMAPS to control access to the service.

By default, machine-types/vobox now restrict login to the VO box to VO software managers and production role. See gLite customization for more details.

gLite-3.1.0-7: Torque directPaths don't depend on shared home directories

In previous QWG releases, WN_SHARED_AREAS entries were not configured as Torque direct paths (configured with $usecp directive in MOM client configuration) when the CE was not configured to use shared home directories. This has been fixed and now every entry in WN_SHARED_AREAS is configured as a Torque direct path unconditionally. Existing WN configuration may change but this should be harmless.

gLite-3.1.0-7: stricter checks in filesystem/config.tpl may lead to compilation errors

As described in release notes, the standard configuration template in charge of producing disk partition configuration based on a layout descrption now implements much stricter validation checks on the configuration. As a result, existing layout may need to be fixed. Two usual problems are:

The disk partition referred in a device or devices attribute is not on the right disk or is using a partition already used by another file system or block device.
The variable in your site layout equivalent to DISK_GLITE_PARTS in the layout examples is not containing an entry for the extented partition in fourth position but there is the need to create a logical partition to implement your layout.

See the documentation for more information about producing a layout template.

gLite-3.1.0-6: DPM major upgrade requiring a database upgrade

QWG release 3.1.0-6 contains DPM 1.7.0. Upgrading to this version requires a database upgrade. As this operation may take some time depending on your DPM database, this is recommended to plan it carefully and declare a downtime for the SE. See release note.

gLite-3.1.0-5: VOBOX update requires a manual removal of one RPM

Update of any VOBOX with new gLite updates will fail during RPM installation with rpmt complaining with the following error:

 depcheck: package glite-VOBOX 3.1.22-0 needs glite-security-lcas-lcmaps-gt4-interface >= 0.0.14-2

This is due to something in the glite-VOBOX metapackage, described in GGUS #46947. Until this problem is fixed, to successfully update a VOBOX, you need to issue the following command before deploying the update:

rpm -e --nodeps glite-security-lcas-lcmaps-gt4-interface

If you already deployed the update and got the error, after deleting the RPM with the previous command, restart ncm-cidspd (or reboot the machine):

service ncm-cdispd restart

gLite-3.1.0-4: BDII configuration change requires a manual restart

3.1.0-4 of QWG templates brings last version of BDII, as introduced by gLite update 33. With this new version, BDII var has been relocated from /opt/bdii/var to /var/bdii. Because of this configuration change, BDII cannot be properly stopped. After deploying the upgrade, this is necessary to stop manually all BDIIs and restart them. This applies to resource BDIIs, site BDIIs and top BDIIs. Resource BDII runs on almost all machine types except WN and UI.

To stop BDII:

service bdii stop
kill -TERM  all slapd processes

To start BDII:

service bdii start

gLite-3.1.0-4: SITE_OTHER_INFO value may prevent successful compilation

In previous releases, SITE_OTHER_INFO used to be a list of strings whose values were up to the site. In accordance with a more formal definition of the corresponding attribute in Glue schema (GlueSiteOtherInfo), this variable is now internally converted to a nlist when specified as a list. For each value, if they contain an '=' character, characters before the = are used as a nlist key and those after as the value. If there is no = the whole value is considered as a key and the value is empty.

As a result, the characters used as a key must be valid for PAN nlist key. This is restricted to alphanumeric characters and underscore (- and . are invalid). And the resulting key must not begin with a number.

If the default result is invalid, edit your variable SITE_OTHER_INFO to be a nlist with valid keys. Note that this variable, except for very specific case, should be restricted to values agreed in your grid project.

gLite 3.1.0-4: error if CE_CLOSE_SE_LIST and SE_HOST_DEFAULT are undefined

If both CE_CLOSE_SE_LIST and SE_HOST_DEFAULT variables are undefined, templates won't compile successfully with an error:

      [panc] variable named 'se_hosts' is undefined

This is due to a typo in default/glite/config.tpl fixed in release 3.1.0-5.

gLite 3.1.0-4: potential deletion of VO sofware areas if they are located in SW manager home directory

QWG release 3.1.0-4 fixes a bug where VO accounts other than VO pool accounts where not cleaned up of old files produced by jobs. This includes in particular accounts used for VO software manager role and production role. After some time, this may lead to non working CE either because home directory area is full or because there are too many files or directories in the home directory (or one its sub-directory).

Unfortunately a check is missing to ensure that a home directory is not matching a VO software area. Even this is no longer the default configuration, it used to be a common configuration some years ago. In this case, the bug fixed leads to all files older than 2 weeks in the VO software area being deleted. This is fixed in release 3.1.0-5.

In the meantime, there is two possible options if your site is using such a configuration: it is important to apply one of them to prevent the cleanup to occur on these accounts.

Install the fix available in 3.1.0-5 either by upgrading to last version in the branch, or by replacing grid/glite-3.1/lcg/ce/config.tpl by the version in the trunk. This is the recommended option.
If you cannot afford to apply the fix, disable the processing of accounts other than pool accounts. This can be done by editing grid/glite-3.1/lcg/ce/config.tpl, looking for definition of variable CE_CLEANUP_ACCOUNTS_CONFIG and replacing it with:
```
variable CE_CLEANUP_ACCOUNTS_CONFIG = '';
```

It is recommended to change your configuration to use a home directory for VO software managers distinct from the VO software area. This is easily done by removing the definition of variable VO_SWMGR_HOMES in your gLite site parameters but require some manual steps to ensure ncm-accounts will not move the VO software area to the new home directory, as there is no configuration option to prevent this. The exact steps to follow may vary depending your exact configuration, in particular depending if the current home directory name should remain the home directory or is the VO software area path:

Always start by renaming the home directory of every VO software manager account to something different to ensure that ncm-accounts will not rename it when reconfiguring accounts. If you want to change the VO software area path rather than the software manager home directory path, use the new name for the software area path. This has to be done for every VO supported with a software manager.
If you choosed to rename/move (moving a SW area can be very long and require a downtime) the software area, update the SW_AREAS variable in your site parameters to reflect the new path.
If you choosed to change the home directory for software managers, remove the definition of variable VO_SWMGR_HOMES in your gLite site parameters or update it to reflect the new path if this is not under the same directory parent as VO pool accounts (as defined in variable VO_HOMES).
After deploying your changes, recreate the home directory for software manager accounts, using command usermod. This has to be done for every VO supported with a software manager.

Main Changes

Note : information in section, may refer to a not yet announced release. These informations are related to changes already present in the development trunk and about to be available or already in gLite-3.1 branch.

gLite-3.1.0-10: AII Kickstart plugin default configuration improved

Release 3.1.0-10 of QWG Templates delivers AII Kickstart plugin (aii-ks) version 1.1.33. This new version contains signficant improvements to the default configuration of disks in Kickstart configuration file generated by aii-shellfe from a machine profile. See AII documentation for details.

gLite-3.1.0-8: initial support of CREAM CE

Release 3.1.0-8 of QWG Templates introduces support of the CREAM CE. See the documentation for more information about how to configure it with QWG Templates.

During the initial installation, you may experience a few issues getting the CE to start. See for more information.

gLite-3.1.0-8: ncm-named v2 requires explicit definition of start property

Release 3.1.0-8 of QWG Templates delivers the last version of ncm-named, v2. This version is a complete rewrite of the previous version to fix several issues and unexpected behaviours. It is intended to be backward-compatible with one exception concerning the behaviour of the component if start property is undefined.

In v1.x version it was true by default, with the potentially unexpected behaviour that named was started (with the existing configuration, whatever it was) as soon as the RPM was installed. undefined was considered the same as false and in this case, an enabled/running named was disabled/stopped.

In the new version, it is start is undefined by default. But when undefined, ncm-name does nothing to a currently enabled/running named (managed by some other means).

If you happen to rely on the old unexpected behaviour, please update your ncm-named configure to add:

'/software/components/named/start' = true;

gLite-3.1.0-8: VOMS server customization improved

Release 3.1.0-7 of QWG Templates introduced the the support of VOMS server but this initial support was lacking a default configuration for the VOMS server parameters. It was thus necessary to add them as part of the site configuration through undocumented variables.

Release 3.1.0-8 of QWG Templates adds standard and documented variables to define the site-specific VOMS server configuration. In case, you previously defined the missing variables, you need to update your site configuration as the change is not backward compatible.

Also note that the default VOMS server configuration is using userid tomcat for the Apache Tomcat configuration used by VOMS server instead of the default tomcat4 for other gLite services relying on tomcat like MONBOX.

gLite-3.1.0-8: Tomcat user configuration improvement

QWG templates used to create and configure a user called tomcat4 for Apache Tomcat. This user is potentially conflicting with the user tomcat created at Tomcat installation by the RPM. In release 3.1.0-8 of QWG Templates, the user created for Tomcat is still tomcat4 by default for backward compatibility but this can be changed if needed by defining variable TOMCAT_USER to the userid you want to use (for example tomcat). Other account characteristics will be adjusted based on the userid.

gLite-3.1.0-8: VO configuration rewritten

Release 3.1.0-8 of QWG Templates introduces a new version of the functions used to configure VOs on a machine. This was necessary to overcome a few problems in the previous version, in particular in the ability to control the accounts created based on the FQAN enabled, and to improve the support for pool accounts for specific FQAN. See documentation on VO configuration for more details.

In addition, a change has been implemented in the UID allocation for specific FQANs. Previously, it was based only on the order of declaration in the voms_roles resource. In the new version, SW manager and production user are assigned a fixed UID, whatever their rank in the list: SW manager is assigned the first UID in the VO range, production user the second. This should reduce UID changes when there is a change in the list of roles declared. Unfortunatly, during the upgrade, a few VOs have some roles changed. This is mainly: biomed, compchem, dzero, esr, fusion, flast.org, planck. For these VOs the existing potentially conflicting accounts must be deleted. See known issues for more details on possible solutions.

gLite-3.1.0-8: Publishing GlueSite object in a non-standard BDII branch

Starting with release 3.1.0-8 of QWG Templates, it is now possible to publish the GlueSiteobject in a non-standard branch, rather than in mds-vo-name=resource. This is useful if you use several site BDIIs to increase service reliability. With the previous configuration, it was possible to have the GlueSite object published by several site BDIIs. This was harmless if the object was the same with the same DN. It was mainly a problem when using an internal hierarchy of site BDIIs (subsite BDIIs) inside the site.

The new feature allows not to mix the GlueSite object with the other objects published by the resource BDII of the site BDII and thus to get it published to the top BDII only by the active site BDII (the one used by the DNS alias associated with the service). See BDII configuration documentation for more details.

gLite-3.1.0-8: Profile cloning improved

Profile cloning, formerly known as dummy WN, has been rewritten in release 3.1.0-8 of QWG Templates to improve its configuration flexibility and consistency. The configuration method is intended to be backward compatible and is described in gLite-related documentation.

Examples have been added into examples provided with QWG templates.

gLite-3.1.0-8: New version of DPM/xrootd

A new version of xrootd for DPM, 20090729.0855-2, is provided starting with gLite update 58 in release 3.1.0-8 of QWG Templates. This new release fixes all the known issue with the previous, very old, release of xrootd shipped with DPM.

Some configuration inconsistencies have been fixed in DPM/xrootd and this requires to remove the following options from your DPM configuration if they were defined:

/software/components/dpmlfc/options/dpm/xroot/config
/software/components/dpmlfc/options/dpm/xroot/ofsPlugin

The default value for these options is now appropriate.

Due to a daemon name change, the upgrade may involve a manual step. See know problems.

gLite-3.1.0-7: Torque version for i386 updated to be consistent with x86_64

Starting with gLite update 54, Torque version used on i386 machines is upgraded to 2.3.6 to be consistent with x86_64 version. It used to be an older version on i386 but this may lead to Torque instabilities in some circumstances, in particular if the server is older than clients.

gLite-3.1.0-7: OS_VERSION_PARAMS added

Standard configuration of a gLite machine now defines as part of the OS selection a new variables OS_VERSION_PARAMS to ease further processing based on OS version (major or minor) or architecture. See OS configuration for more details.

gLite-3.1.0-7: improved OS errata management

Release 3.1.0-7 of QWG Templates has an improved support for OS errata, offering an increased flexibility in large environments where you want to use a stage deployment. See the documentation for more details.

gLite-3.1.0-7: disk partitionning enhancements and fixes

The template that allows configuration of disk partitions based on a layout template has been fixed to handle properly configurations not based on LVM (extended partitions and software raid). It has also been improved to do much more validation checks on layout description and do the necessary renumbering of partitions to ensure numbers are consecutive and that partitions are created in the appropriate order (for example, partitions without an explicit size created last).

Layout examples are now provided for the 3 major configurations: LVM-based, extended partitions and software raid.

As a result of these stricter checks, your existing may not compile successfully anymore. See known problems.

gLite-3.1.0-7: MyProxy configuration improved to support fine-grained policies

Until 3.1.0-7, MyProxy configuration supported only one category of clients that were all configured as both renewers and retrievers. This has been improved to give a finer control and more flexibility on how MyProxy is configured. See https://trac.lal.in2p3.fr/LCGQWG/wiki/Doc/gLite/TemplateCustomization#MyProxyServer.

gLite-3.1.0-7: Torque mom and scheduler removed from Torque server configuration

torque-mom, torque-scheduler and torque-localhost have been removed from Torque server configuration. If torque is not used with MAUI and you want to use Torque native scheduler, define variable TORQUE_USE_PBS_SCHED to true.

gLite-3.1.0-6: SITE_GLOBUS_TCP/UDP_RANGE replaced by GLOBUS_TCP/UDP_PORT_RANGE_MIN/MAX

In previous releases of QWG templates, Globus ephemeral port range was specified using variables SITE_GLOBUS_TCP_RANGE (TCP) and SITE_GLOBUS_UDP_RANGE (UDP) . Both variables should specify a range with the appropriate delimiter for the Globus version used. This has been replaced by 2 variables for each range specifying the lower and upper port respectively.

If defined, SITE_GLOBUS_TCP_RANGE (TCP) and SITE_GLOBUS_UDP_RANGE (UDP) are honoured and take precedence over the new variables but they are no longer defined by default.

See documentation about gLite customizations for more details.

gLite-3.1.0-6: xroot support in DPM configuration

QWG templates had some sort of preliminary support for xroot access protocol configuration in DPM. It has been entirely rewritten, integrated into the component ncm-dpmlfc managing other parts of DPM configuration. It supports both unauthenticated access and token-based authentication. See sites/example/site/glite/dpm_config.tpl for an example.

gLite-3.1.0-6: generic OS flavour optionally added to load path

When variable OS_FLAVOUR_ENABLED is true, another OS related path for looking for templates is added to the default load path. Its name is derived from OS version, with the update number replaced by 'x'. For example, for sl460-x86_64, this will be sl4x-X86_64. And this will be the same for any SL4 x86_64.

This feature is disabled by default.

gLite-3.1.0-6: update 46+ implies a major DPM upgrade

gLite update 46, integrated into QWG release 3.1.0-6, includes version 1.7.0 of DPM. Upgrading to this version requires and upgrade of DPM database.

It is possible to postpone DPM upgrade when upgrading to QWG release 3.1.0-6 by defining variable GLITE_UPDATE_VERSION to 45 in profiles of DPM nodes.

To upgrade to DPM 1.7.0, the recommended step-by-step procedures is:

Deploy update 46 or later by undefining variable GLITE_UPDATE_VERSION
Stop all DPM daemons on DPM head node (dpm, dpnsdaemon, srmx...)

Upgrade DPM database with the following command after putting the DPM database access password into /tmp/dpmpwd:

  nohup ./dpm_db_310_to_320 --db-vendor MySQL --db localhost --user root --pwd-file /tmp/dpmpwd --dpm-db dpm_db --verbose >/tmp/dpm_db_upg.log 2>&1

Check in /tmp/dpm_db_upg.log that the upgrade was successful and restart daemons (or reboot)
Restart DPM daemons on DPM disk servers (or reboot them)

Note: it is recommended to do a backup of the DPM databases before proceeding with the upgrade.'

Should you have the error The total number of locks exceeds the lock table size, look at GridPP blog to know how to fix it and rerun the upgrade. To avoid this error and improve performances, if you have enough memory on your DPM head node, it is recommended to adjust MySQL InnoDB parameter as suggested before starting the upgrade (and keep it in your production configuration).

gLite-3.1.0-6: HW configuration used to compute published number of CPUs per CE

Previously, the number of published CPUs per CE was based on variable WN_CPUS and WN_CPUS_DEF. In addition GlueSubclusterLogicalCPUs and GlueSubclusterPysicalCPUs were not publishing the values as clarified in the last year where:

GlueSubclusterPysicalCPUs is the number of sockets per machine, whatever number of cores per socket.
GlueSubclusterLogicalCPUs is the number of execution units (cores) as seen by the operating system (/proc/cpuinfo).

Both of these values must reflect the installed capacity and not the number of active nodes. So they should not be adjusted based on the active configuration.

This has been changed to compute the number of physical and logical CPUs, based on HW configuration of worker nodes listed in variable WORKER_NODES. Variables WN_CPUS and WN_CPUS_DEFare now ignored, except if the information is missing in HW description of the worker node. This change may require some adjustment in HW configuration of worker nodes. Historically, CPU configuration described in HW templates was creating one CPU entry per core. This must be changed to be one CPU entry per socket and the template describing the CPU must have the property cores defined to the number of cores per socket.

gLite-3.1.0-6: update/unofficial no longer included by default

Unexpectedly update/unofficial was always added as part of standard configuration of gLite updates. This is no longer the case. To add it, variable GLITE_UPDATE_UNOFFICIAL must be defined to true.

gLite-3.1.0-6: new globus-gma (LCG CE) added

As part of configuration of gLite 3.1 update 42, a new version (1.0.11) of globus-gma has been provided for the LCG CE. This new version has not yet been officially released but offers significant improvements for large CEs, in particular for handling short jobs.

Note: since then, this version (and newer versions) of globus-gmahave been officially released.

gLite-3.1.0-5: add support NFS v4

3.1.0-5 of QWG templates introduces support for NFS v4. See gLite customization for details.

gLite-3.1.0-5: new WMS version requires a manual conversion step

3.1.0-5 of QWG templates delivers new WMS version (glite-WMS 3.1.12-0) released as part of gLite update 41. There is a change in the internal format of one of WMS queues. Before upgrading the WMS, it is recommended to drain it, to avoid acceptance of new requests during the upgrade.

If you don't want to wait for all jobs to be completed before upgrading the WMS, it is possible to convert existing queue after deploying the new version. This is easily done with the following command:

/opt/glite/sbin/glite-wms-fl2d /var/glite/workload_manager/input.fl /var/glite/workload_manager/jobdir

After the conversion, input.fl can be deleted. If you use non standard path, update the paths in the previous command.

gLite-3.0.1-5: NFS configuration rewritten to increase flexibility

NFS configuration has been entirely rewritten to improve flexibility and add new configuration options. NFS client and server configuration is now part of machine-types/base.tpl used by all the other machine types. As a result every machine type can also be configured as a NFS server or client. This is controlled by the variables NFS_SERVER_ENABLED and NFS_CLIENT_ENABLED that are autoconfigured based on machine types and variable WN_SHARED_AREAS contents. See gLite customization for more information.

A new feature introduced is the ability to relocate home directories for VO accounts to a NFS file system when default is /home. This allows to keep home directories for service accounts on a local file system : this is important to avoid side effects on other services when NFS is unresponsive. This is done using variable VO_HOMES_NFS_ROOT.

If you built your own site-specific machine types copying machine-types/base.tpl provided with gLite templates and still relying on NFS configuration templates, you may need to update your private template to implement the same changes as in gLite machine-types/base.tpl. The typical sequence to configure NFS is:

common/nfs/init;
common/nfs/server/config;
common/nfs/client/config;

Also note that if you are using the standard template for disk partionning, standard/filesystem/config.tpl, it must be called after common/nfs/init.tpl.

gLite-3.0.1-5: GlueHostArchitecturePlatformType correctly published

GlueHostArchitecturePlatformType is now correctly published (it was unset in previous version). It is currently set to the architecture of the CE by default, even though it should relect WN architecture. This will be improved when it will be possible to have several subclusters per cluster. If not appropriate, default value can be changed by defining CE_WN_ARCH varialbe in gLite site parameters to the appropriate value.

gLite-3.0.1-5: ability to group home directories per VO

In previous releases of QWG templates, the only possibility to have a different parent for home directories of each VO was to add an explicit entry for each VO in variable VO_HOMES. But when using the same file system for all home directories, it may still be desirable to have one subdirectory per VO to avoid performance problems resulting from thousands of entries in the same directory.

QWG release 3.1.0-5 introduces to special keywords that can be used when defining home directory parents: @VONAME@ and @VOALIAS@, expanded respectively into the the VO full name or the VO alias defined locally.

gLite-3.1.0-4: support for VOS='ALL'

On a UI it may be suitable to allow users to act as a member of any VO they belong too, not only a limited number of VOs supported by local resources. This is now possible defining variable VOS as string 'ALL' (instead of a list). This results on VO configuration being added for all the known VOs in the configuration (normally all VOs registered CIC portal). It is strongly recommended not to use this configuration on a machine other than a UI. On a gsissh-enable UI, be sure to restrict the number of VOs enabled to use gsissh to a limited number, in particular as gsissh support requires creation of VO accounts.

Refer to gLite customization for more details on how to use this option and the known restrictions.

gLite-3.1.0-4: support added for configuration of Nagios (server and client)

Release 3.1.0-4 of QWG templates provides an integrated support for configuring Nagios server and clients with Quattor.

gLite-3.1.0-4: preliminary support added for openVZ

Release 3.1.0-4 of QWG templates provides the initial support for virtualization with openVZ in SL 5.1 64-bit and later.

gLite-3.1.0-4: dummy WN speed up integrated to standard templates

In release 3.1.0-4 of QWG templates, the dummy WN speed up has been integrated into standard templates. This option allows to compile only one WN (a reference WN) and include in other WNs the already compiled reference WN. The compiled reference WN configuration is then tweaked to adjust network parameters and a few others to the actual WN. This may result in a very significant speed up for configuration with a large number of WNs.

Refer to gLite customization for more details on how to use this option and the known restrictions.

gLite-3.1.0-4: NFS default configuration changes and improvements

NFS configuration has been improved to provide greater flexibility when configuring different kind of machines with the same configuration information and to tighten security. In addition default mount options on clients now include noatime to improve performances. This is particularly important for sites with a large number of NFS clients (for example, WNs).

Configuration changes related to security include enabling of root squashing on all file systems, except /home, by default. It is also possible to define the list of allowed clients (used to configure the exports) on a per-filesystem basis.

For details about the new configuration options, refer to gLite customization.

gLite-3.1.0-4: GlueSiteOtherInfo default maching EGEE requirements

In accordance with a more formal definition of attribute GlueSiteOtherInfo (defined in Glue schema used by BDII) by EGEE, it now has a default value matching EGEE requirements. Additional key/value pairs can be defined using variable SITE_OTHER_INFO which must be preferably defined as a nlist (it used to be a list). When it is defined as a list, it is internally converted as a nlist using = character in values if any to split the value in a key and a value (else it is converted as key with an empty value).

In addition to the default value, a site should define at least key EGEE_ROC in SITE_OTHER_INFO. This must match the name of the EGEE ROC the site belongs to. And if it is a WLCG site, it must define additional keys specifying its role (Tier) and the T1 it is attached too. Look at example of site parameters for more details.

If the default value is not matching your requirements, you can undefine it in your site parameters with:

variable SITE_OTHER_INFO_DEFAULT ?= undef;

gLite-3.1.0-4 : Torque default server attributes

In release 3.1.0-4 of QWG templates, default value for log_file_max_size has been defined to 200 MB instead of 0, as interpretation of 0 is broken and cause the log file to be rolled every 5 mn.

In addition this is now possible to undefine an attributed defined by default by assigning it value undef in variable TORQUE_SERVER_ATTRS. See gLite customization for details.

gLite-3.1.0-4 : Torque queue configuration change to support long VO name

In previous versions, configuration of Torque queue was relying on CE_QUEUES variable that had to be defined in gLite site parameters. This variable had to contain an entry for each of the queue to be created (generally one per VO).

In release 3.1.0-4 of QWG templates, this has been changed to improve support of VO names that are not a valide queue name for Torque, for example VO names longer than 15 characters. gLite site parameters must now define variable CE_QUEUES_SITE instead and this variable must have an entry only for queues with non standard or site specific parameters. The standard queue names are generated by Torque configuration templates from the the VO names. Example for gLite site parameters has been updated to use the new variable. In addition it is possible to delete a queue created by default by defining its attlist to undef in CE_QUEUES_SITE.

For backward compatibility, it is still possible to define directly CE_QUEUES in site parameters and Torque will use it without any modification. In this case, the site must take care of generating Torque compatible queue names.

gLite-3.1.0-4 : default name for VO SW area changed to VO name

In previous version of the QWG templates, SW areas created using the DEFAULT entry in VO_SW_AREAS had a name based on SW manager userid for the VO. This has been changed to be the VO name (VO full name even if a VO alias is used) by default and old behaviour can be restored defining variable VO_SW_AREAS_USE_SWMGR to true.

gLite-3.1.0-4 : syntax of templates compliant with PAN v8

PAN compiler v8 introduces some syntax changes. A few valid construct in previous versions are now marked as deprecated and will not be supported anymore in v9. All templates have been made compliant with the new syntax, removing all deprecated constructs. The main changes are:

include now requires a DML: include mytempl; must be replaced by include {'mytempl'};. This is to prepare changed planned for v9.
All automatic variables like self, object, argv, argc must now be capitalized.
type is no longer a valid keyword to bind a path to a type: bind must be used instead. type is now restricted to type definitions.

gLite-3.1.0-4 : DPM and LFC 1.6.10 delivered as part of update 21

gLite 3.1 update 21 as provided by QWG release gLite-3.1.0-4 delivers DPM and LFC 1.6.10 instead of the standard 1.6.7-4. This version has been certified but not officially released. Despite this, this version is believed production ready and brings new utilities for managing space reservations (dpm-getspacexxx) and some additional fixes, in particular for command dpm-drain.

gLite-3.1.0-4 : New configuration scheme for Torque/MAUI

QWG release gLite-3.1.0-4 introduces a more flexible scheme for Torque and MAUI configuration. Previously, site-specific Torque customization was difficult and MAUI configuration had to be provided in extenso in MAUI_CONFIG variable.

New scheme is based on a set of variables described in gLite customization documentation. To use this new configuration scheme for MAUI, you need to undefine MAUI_CONFIG in your site parameters and replace site-specific parameters with appropriate variables. MAUI configuration example has been updated.

gLite-3.1.0-4 : Support for Torque/MAUI 64-bit

QWG release gLite-3.1.0-4 introduces use of Torque/MAUI 64-bit on 64-bit OS by default. If you want to keep running version 32-bit, you need to define the following variable in your CE profile (and WNs) :

PKG_ARCH_TORQUE_MAUI ?= 'i386';

There is no problem mixing Torque/MAUI 32-bit and 64-bit in the same cluster.

Note: when upgrading from a 32-bit version to a 64-bit version, the Torque server must have been drained (queues must be empty) as /var/spool/pbs/server_priv/serverdb created by 32-bit Torque cannot be read by 64-bit Torque and vice-versa. This file must be removed before starting 64-bit Torque version.

gLite-3.1.0-4 : AII upgraded to v2, new disk partitioning scheme

QWG release gLite-3.1.0-4 delivers AII v2. This new version of AII, used for initial installation, brings much more flexibility in definition of file system and block devices and in ability to specifiy site-specific actions during installation. In particular, former Kickstart template has been replaced by hooks that are much easier to specify. You can find some documentation about new AII configuration and migration issue in the page about initial installation.

In addition to AII v2, a new standard template has been introduced to define partitionning. Even though this template cannot probably match all configurations, it has been designed to be pretty flexible and allow quie a lot of different configuration strategies. When using this templates, a default partitionning can be defined and futrther refined for a specific machine using a few variables. Look at AII documentation for more details.

The price for this new flexibiliy is that non backward-compatible changes are required in site configuration. This part of the configuration is normally not used after initial installation, so there is no real risk of breaking a working configuration. The main problem will be difficulties to install new machines or reinstall old ones.

gLite-3.1.0-3 : FTS Client Configuration Improvement

In QWG release gLite-3.1.0-3, FTS client configuration has been reworked to be simpler and more flexible. The main variable to configure the FTS client is now FTS_SERVER_HOST. Look a gLite customization for more details.

gLite-3.1.0-3 : LCG CE added

QWG release gLite-3.1.0-3 add support for gLite 3.1 LCG CE. LCG CE service can be installed both on 32-bit and 64-bit platforms.

Note : LCG CE in gLite 3.1 is no longer configured as a site BDII by default. If you use your CE as your site BDII (not recommended), you need to define variable BDII_TYPE to site in your CE node profile.

Note : In LCG CE 3.1, Globus job manager lcgpbs is no longer available. As a result, a shared file system for home directories is required (job manager pbs doesn't work with home directories on a non shared filesystem). If you were using job manager lcgpbs, you need to edit your site parameters and change it to pbs (or use default). Be aware that this change implies a changed in contact string for the CE as this contact string contains the job manager name. This is transparent for job submitted through a RB/WMS (if the CE was drained before the change was done) but may require a configuration update for framework which uses direct submission to the CE (.e.g. Atlas PanDA).

gLite-3.1.0-3 : MyProxy added

QWG release gLite-3.1.0-3 add support for gLite 3.1 MyProxy. MyProxy service can be installed both on 32-bit and 64-bit platforms.

gLite-3.1.0-3 : BDII added

QWG release gLite-3.1.0-3 add support for gLite 3.1 BDII. MyProxy service can be installed both on 32-bit and 64-bit platforms.

gLite-3.1.0-3 : LFC 32-bit and 64-bit added

Support for official release of LFC on SL4 (LFC 1.6.8), both 32-bit and 64-bit native build, has been added in release gLite-3.1.0-3. In addition, it is now possible to configure LFC aliases, see template customization.

Due to a bug in GIP provider for LFC, this is not possible to run a site BDII on LFC server (see GGUS #31741, fix planned in LFC 1.6.10).

You can upgrade directly from SL3 36bit to SL4 32-bit but upgrade to SL4 64-bit (recommended) requires a reinstallation of the machine. This can be done safely after backuping MySQL database (mysql-dump) and saving log files (they must be kept 90 days).

gLite-3.1.0-3 : DPM 32-bit and 64-bit added

Support for official release of DPM on SL4 (DPM 1.6.7), both 32-bit and 64-bit (not yet supported by YAIM) native build, has been added in release gLite-3.1.0-3. Disk servers using this version can be used in a DPM configuration with gLite 3.0 nodes running DPM 1.6.5.

You can upgrade directly from SL3 36bit to SL4 32-bit but upgrade to SL4 64-bit (recommended) requires a reinstallation of the machine. On DPM head node, this can be done safely after backuping MySQL database (mysql-dump) and saving log files (they must be kept 90 days). On DPM disk servers, you need to ensure that DPM partitions will not be affected by reinstallation. You can check DPM upgrade procedure for YAIM.

gLite-3.1.0-3 : Java version rolled back to 1.5

Due to problem integrating version 1.7 of JPackage, required by gLite 3.1 to provide some external packages, with Java 1.6 and impossibility to install 1.5 if 1.6 is already installed, Java version provided by SL4.4 and SL 4.5 templates has been rolled back to 1.5.

gLite-3.1.0-3 : Improvements to configuration of RPM repositories

Release gLite-3.1.0-3 of QWG Templates brings more flexibility in configuration of RPM repositories. In previous versions, a site had to provide 3 RPM repositories per gLite version (release, externals, updates) with fixed names or had to edit the standard template repository/config/glite.tpl.

There is now limited possibilities to tune the standard configuration without editing the standard templates :

Define the repository prefix used for all the repositories of a particular gLite version using variable REPOSITORY_GLITE_PREFIX. Default is glite_3_1.
Ability to define a site specific repository associated each gLite version by providing a repository template whose name is repository_prefix_unofficial.tpl.

See gLite template customization for more details.

gLite-3.1.0-2 : GLOBUS_TCP_PORT_RANGE syntax change

GLOBUS_TCP_PORT_CHANGE now uses GT4 syntax (min and max value separated by a comma instead of a space) and extra quoting in the shell environment variable has been removed.

gLite-3.1.0-2 : DPM GSIFTP missing file fixed

Missing .templ file for DPM GSFTP has been fixed.

gLite-3.1.0-2 : UI upgrade oddities

If upgrading a UI from gLite 3.0 to gLite 3.1 (without reinstalling), it is necessary to reconfigure EDG RB client as one RPM provided by the update overwrite some configuration files for it. This is easily done with :

ncm-ncd --configure wmsclient

Symptom of an overwritten configuration is that command edg-job-submit contact the wrong logging server.

Also be aware that VDT uprade issue described for WN also applies to UI. Follow the instruction given for the WN.

gLite-3.1.0-1 : configuration files missing for DPM

DPM disk server RPM is buggy and the template for one configuration file is missing. As a result, gsiftp daemon is silently misconfigured. To fix this issue you need to create file `/etc/sysconfig/dpm-gsiftp.templ by copying the file from a previous version on another disk server or by manually creating it with the following contents :

#
# Please change the following values according to your setup :
#

#
# DPM Name Server host : !!!!! please change !!!!!
#
export DPNS_HOST=<DPNS_hostname>

#
# DPM Server host : !!!!! please change !!!!!
#
export DPM_HOST=<DPM_hostname>

Then you need to reconfigure DPM with the following command :

ncm-ncd --configure dpmlfc

gLite-3.1.0-1 : issue upgrading a WN from 3.0 to 3.1

There is an issue preventing the successful upgrade of a WN from gLite 3.0 to 3.1 and requiring a specific action. The RPM externals/gpt-VDT upgrade fails because of a conflict with the previously installed version. Unfortunatly, this problem doesn't lead to a RPM error status and it can only be asserted by executing the following command :

rpm -q gpt

If gpt version is 1.2.2, the upgrade failed. The problem can also be confirmed by looking at /var/log/spma where there should be the following error message :

error: unpacking of archive failed on file /opt/gpt/share/globus_aclocal: cpio: rename failed - Is a directory

For the upgrade to successfully complete, it is necessary to delete or rename the two following directories :

mv /opt/gpt/share/globus_aclocal /opt/gpt/share/globus_aclocal-3.0-to_delete
mv /opt/gpt/share/gpt_amdir /opt/gpt/share/gpt_amdir-3.0-to_delete

Note : starting with release gLite-3.1.0-2, Quattor does it during the upgrade if the upgrade failed.

After the upgrade, you need to check each upgraded machine and, if gpt has not been upgraded, to enter the following command :

service ncm-cdispd restart

gLite-3.1.0-1 : components migrated to namespace

This release updates most of the components to the last version, in order to use namespaced version of the components. This changes the template name used to include to component in the configuration. Most of the changes are internal to QWG templates but some components are also included in site templates. The change is pretty straighforward : pro_sofware_component_xxx must be replaced by components/xxx/config.

Look at gLite-3.0.2-12 release notes for more information on how to automate edits.

gLite-3.1.0-1 : repository templates migrated to namespace repository/

In this QWG release, templates related to RPM repositories configuration have been migrate to namespace repository/, to match the agreed convention. In addition, the templates have been renamed withouth the leading repository_lal or repository_common prefix.

repository/ namespace now has the following structure :

config.tpl is the new name for the per-cluster repository_common.tpl that defines the RPM repositories used in the cluster. Look at example.
config/ sub-namespace contains the repository used by a specific part of QWG templates. Normally this namespace is used mainly in standard, grid, or os (SL 4.4 i386 example).
Other .tpl files in repository/` defining the repository contents. These templates are generally locating in the site specific templates. Look at examples.

This change is not completly backward compatible, as it requires updating repository/config.tpl (former repository/repository_common.tpl) to use the new names.

You are advised to update your configuration to adhere to this standard naming scheme, as it should allow smoother upgrades in the future. But if you want to ignore this change in a first stage, you can revert changes affecting repository/*.tpl templates : this way you should be able to install this release without changes in your site configuration.

To update your configuration to use the new naming scheme for repositories, you need :

Rename your repository templates (probably in your site template hierarchy) to adhere to new repository template names with svn mv.
Remove everything in these templates after the initial comments and execute ant update.rep.templates.
Check and if necessary update cluster.build.properties for each of your clusters : be sure to have the namespace form of your site directory in the include path. Look at cluster example.

If you are using SWrep to manage repositories and repository templates, upgrade to a version supporting repository namespace (part of Quattor 1.3).

For more information about repository templates used by the default repository/config/glite.tpl, look at gLite templates customization.

Note : be sure to use SCDB 2.1.2 or later when upgrading to new naming scheme. There is a known issue with previous versions.

gLite-3.1.0-1 : CE_BATCH_SYS=lcgpbs is no longer supported

gLite 3.1 doesn't support anymore lcgpbs as batch system type. You need to change it to pbs if you are still using lcgpbs. This requires to drain the CE before doing the change.

pbs batch system type provides all the features that were available with lcgpbs except support for non shared home directories but brings the support for MPI jobs.

gLite-3.1.0-1 : SL 4.4 required

QWG release gLite-3.0.1-1 requires SL 4.4. There is no plan to support older release of the operating system.

WN support both 32-bit and 64-bit versions of the OS. DPM disk server requires 64-bit architecture.

gLite-3.1.0-1 : supported gLite services

QWG release gLite-3.0.1-1 provides only 2 gLite services :

WN officially released as gLite 3.1
DPM disk server : not yet released officially, based on the upcoming DPM 1.6.5. It is already in use in production at several sites.

gLite-3.1.0-1 based on gLite-3.0.2-12

QWG release gLite-3.0.1-1 is based on a gLite-3.0.2-12 release. Most of the changes described in templates for gLite 3.0 release notes apply to gLite-3.0.1-1, compared to previous gLite-3.0.2 releases.

Change Log

ChangeLog build from repository commit messages...

Changelog not available

Last modified 13 years ago Last modified on Feb 27, 2011, 2:26:46 PM

Download in other formats:

Plain Text