How to Install QWG Templates
TracNav
Table of Contents
Note : QWG templates are intended to work both with SCDB and Aquilon. But their integration has been tested mainly with SCDB and tools described here applies mainly to SCDB. Report every issue that you find on GitHub.
Initial Installation
For an initial installation of QWG templates, you first need to initialize you repository with a base configuration database. If you want to use SCDB, follow instructions on how to configure your Quattor server with SCDB, in particular SCDB initialization.
Upgrading an existing repository
If you are using the recommended template layout with separate areas for standard templates in your configuration database, upgrading should be no more than replacing the previous version of the templates.
We do our best to ensure that new versions are backward compatible. This means that if you made site customization using the described method (pro_lcg2_config_site, pro_site_cluster_info...) and didn't modify standard templates, no changes should be required to use new version of the templates. When this is really required to introduce a new feature or bring more flexibility for configuring a service, it can happen that this full compatibility cannot be retained. This is normally stated in release announcement. You are always better to check it before upgrading.
Standard templates are made of 3 areas (see template layout) :
- OS templates : they are never modified, except for fixes. The only modifications apply to errata RPM list. These errata are not installed by default.
- Middleware templates (if using them) : they bring both changes and fixes at each release. They are intended to be backward compatible : that means the resulting configuration should be the same if you don't change your local configuration.
- Quattor templates: all the templates required to configure a given Quattor release (client, configuration modules...)
- Standard templates : all modifications in this area are entirely backward compatible
There are 2 methods to upgrade QWG templates, according to the structure and constraints of your site :
- In place upgrade : in this scenario, existing templates will be upgraded and the changes will be deployed on all clusters of your site. You can control the gLite updates you deploy on a per node basis.
- Fresh installation of new middleware templates : this allows to control on a per cluster basis the deployement of new QWG templates. In this scenario, you are still doing an in-place upgrade of standard and OS templates.
In-place Upgrade
An in-place upgrade basically involves :
- Getting a copy of the last version of the template library, using the download procedure
- Synchronize your configuration database with your local copy of the template library.
To integrate new/changed templates in your configuration database, the easiest is to use the tool directory-sync. This tool can be used only with SCDB, as it relies on Subversion to do a non destructive update. As a benefit, it cannot damage your configuration database as every change made by this tool can be reverted with Subversion.
This tool updates a directory with the contents of a reference source, doing the following :
- For each directory existing both in QWG repository and in source tree :
- Removes all
.panfiles from the destination directory (the directory to update). - Copy all .pan files from the source directory (built by appending destination directory name to source directory root)
- Removes all
- For each directory in the source tree and not in the repository, copy the tree from the source tree and add files to SVN QWG repository.
- For each directory in the repository but no longer in the source tree, do a
svn rmof the directory. - Show the differences with
svn status
It accepts 2 parameters:
- directory to synchronize: must be a relative path specification.
- reference directory parent: the parent path where a directory with the same relative name is located and must be used as a reference source.
Note: because the second argument is not the source directory itself but a parent for a directory with the same relative name as the first parameter, when synchronizing a real configuration database with a QWG branch or trunk you must run this tool from cfg directory or one of its subdirectories. As you run this tool from a SVN workspace, you always have the opportunity to do a svn revert in case things went wrong...
find grid os standard -type d -not -path '*.svn*' -exec tools/directory-sync {} /tmp/scdb-vanilla/cfg \;
You need to fix manually the unexpected differences. Unexpected differences are all the templates that existed in the repository but no longer exist in the local workspace or all the templates copied from the source but not registered in the repository. There are 3 situations :
- A template has been renamed or moved. In this case, you should revert the change for this template, do the relevant
svn mvand rerun directory-sync to avoid breaking the template modification history. - Else, if the template existed in the repository but is no longer present in the workspace (flagged
!bysvn status), it can be safely removed from the repository with command :# To remove template template_name.tpl svn rm --force template_name.tpl # To remove all templates no longer present in the repository svn rm `svn status | grep '^\!' | awk '{print $2}'` - Else, if the template is present in the workspace but not registered into the repository ((flagged
?bysvn status) and this is a new template, it should be added to the repository with one of theses commands :# Add one specific template template_name.tpl svn add template_name.tpl # Add all the files not yet registered in the repository svn add `svn status | grep '^\?' | awk '{print $2}'`
When you have cleaned up the situation for all the templates, except those that are present in the workspace (because they are used in your local Quattor configuration) but don't need to be in the repository, you can do the final cleanup with :
rm -f `svn status | grep '^\?' | awk '{print $2}'`
When there is no more templates flagged as ! (missing) or ? (unknown), you can commit your changes to the repository. Please, take care of setting up a useful message for the commit as this is the only source for a ChangeLog...
After importing all the changes with directory-sync (or manually), before committing them into your SCDB, check that you can compile them successfully and that the changes in the profiles look appropriate (generally mainly some RPMS update). For this last check you can use src/utils/profiles/compare_profile script. After integrating the new templates, identify the changed templates, review the change if necessary and revert those not appropriate. You'll probably need to revert changes of all templates in directories repository (these templates are site specific but some of them are required to reside in standard directories for multiple version support).
Fresh Installation of New Middleware Templates =
In this scenario, you first have to download the new version of the templates, as in previous scenario. After, rather than sync'ing your templates with the release, you can copy the middleware templates provided by the release (e.g. grid/glite-3.0.0) in a new directory under cfg/grid in your CDB/SCDB and add them to the repository (use svn add for SCDB).
After you can select the deployment of the new release on a per cluster basis. For this, you have to edit cluster.build.properties of the cluster and update the include path, replacing references to previous release of middleware templates by reference to the new one you just added.
Controlling gLite Updates Installation
QWG templates releases deliver the last gLite updates available at the time of the release. There is no equivalent between QWG release number (-n) and gLite update numbers. Sometimes one QWG templates release deliver several gLite updates. In each QWG release, there is a default associated gLite update (generally the last one).
To allow site as much flexibility as possible in selecting what gLite update to install on a specific node, cluster or site, starting with QWG release 3.0.2-10, a release provides each gLite update separatly, not just the last one. You can select the gLite update you want to deploy by defining variable GLITE_UPDATE_VERSION at the beginning of your node profile, in pro_site_cluster_info.tpl' or in gLite site parameters pro_lcg2_config_site` respectively. The value specified must be a string and is generally the gLite update number : in fact it needs to match a directory name under cfg/grid/glite-3.0.0/updates.
For exampple, to stay with update 15 in a specific cluster, edit site/pro_site_cluster_info.tpl for the cluster and add :
variable GLITE_UPDATE_VERSION = '15';
