= How to Install QWG Templates =
[[TracNav]]

[[TOC(inline)]]

''Note : QWG templates are intended to work both with [wiki:Doc/SCDB SCDB] and [wiki:Doc/Aquilon Aquilon]. But their integration has been tested mainly with SCDB and tools described here applies mainly to SCDB. Report every issue that you find on [http://github.com/quattor GitHub].''


== Initial Installation ==

For an initial installation of QWG templates, you first need to initialize you repository with a base configuration database. If you want to use [wiki:Doc/SCDB SCDB], follow instructions on how to [wiki:Download/SCDB#SCDBInitialization configure] your Quattor server with SCDB, in particular [https://trac.lal.in2p3.fr/LCGQWG/wiki/Download/SCDB#SCDBInitialization SCDB initialization].

== Upgrading an existing repository ==

If you are using the recommended [wiki:Doc/TemplateCustom template layout] with separate areas for standard templates in your configuration database, upgrading should be no more than replacing the previous version of the templates.

We do our best to ensure that new versions are backward compatible. This means that if you made site customization using the [wiki:Doc described method] (pro_lcg2_config_site, pro_site_cluster_info...) and didn't modify standard templates, no changes should be required to use new version of the templates. When this is really required to introduce a new feature or bring more flexibility for configuring a service, it can happen that this full compatibility cannot be retained. This is normally stated in release announcement. You are always better to check it before upgrading.

Standard templates are made of 3 areas (see [wiki:Doc/TemplateCustom template layout]) :
 * OS templates : they are never modified, except for fixes. The only modifications apply to errata RPM list. These errata are not installed by default.
 * Middleware templates (if using them) : they bring both changes and fixes at each release. They are intended to be backward compatible : that means the resulting configuration should be the same if you don't change your local configuration. 
 * Quattor templates: all the templates required to configure a given Quattor release (client, configuration modules...)
 * Standard templates : all modifications in this area are entirely backward compatible

There are 2 methods to upgrade QWG templates, according to the structure and constraints of your site :
 * In place upgrade : in this scenario, existing templates will be upgraded and the changes will be deployed on all clusters of your site. You can [wiki:Download/QWGTemplates/Install#ControllinggLiteUpdatesInstallation control] the gLite updates you deploy on a per node basis.
 * Fresh installation of new middleware templates : this allows to control on a per cluster basis the deployement of new QWG templates. In this scenario, you are still doing an in-place upgrade of standard and OS templates.
 
=== In-place Upgrade ===

An in-place upgrade basically involves :
 * Getting a copy of the last version of the template library, using the [/wiki/Download/QWGTemplates download] procedure
 * Synchronize your configuration database with your local copy of the template library.

To integrate new/changed templates in your configuration database, the easiest is to use the tool [source:templates/trunk/tools/directory-sync directory-sync]. This tool can be used '''only''' with [wiki:Doc/SCDB SCDB], as it relies on [http://subversion.tigris.org Subversion] to do a non destructive update. As a benefit, it cannot damage your configuration database as every change made by this tool can be reverted with Subversion.

This tool updates a directory with the contents of a reference source, doing the following :
 * For each directory existing both in QWG repository and in source tree :
   * Removes all `.pan` files from the destination directory (the directory to update).
   * Copy all .pan files from the source directory (built by appending destination directory name to source directory root)
 * For each directory in the source tree and not in the repository, copy the tree from the source tree and add files to SVN QWG repository.
 * For each directory in the repository but no longer in the source tree, do a `svn rm` of the directory.
 * Show the differences with `svn status`

It accepts 2 parameters:
 1. directory to synchronize: '''must be''' a relative path specification.
 1. reference directory parent: the parent path where a directory with the '''same relative name''' is located and must be used as a reference source.

''Note: because the second argument is not the source directory itself but a parent for a directory with the same relative name as the first parameter, when synchronizing a real configuration database with a QWG branch or trunk you must run this tool from `cfg` directory or one of its subdirectories.'' As you run this tool from a SVN workspace, you always have the opportunity to do a `svn revert` in case things went wrong...


{{{
find grid os standard -type d -not -path '*.svn*' -exec tools/directory-sync {} /tmp/scdb-vanilla/cfg \;
}}}

You need to fix manually the unexpected differences. Unexpected differences are all the templates that existed in the repository but no longer exist in the local workspace or all the templates copied from the source but not registered in the repository. There are 3 situations :
 * A template has been renamed or moved. In this case, you should revert the change for this template, do the relevant `svn mv` and rerun [source:templates/trunk/tools/directory-sync directory-sync] to avoid breaking the template modification history.
 * Else, if the template existed in the repository but is no longer present in the workspace (flagged `!` by `svn status`), it can be safely removed from the repository with command :
{{{
# To remove template template_name.tpl
svn rm --force template_name.tpl
# To remove all templates no longer present in the repository
svn rm `svn status | grep '^\!' | awk '{print $2}'`
}}}
 * Else, if the template is present in the workspace but not registered into the repository ((flagged `?` by `svn status`) and this is a new template, it should be added to the repository with one of theses commands :
{{{
# Add one specific template template_name.tpl
svn add template_name.tpl

# Add all the files not yet registered in the repository
svn add `svn status | grep '^\?' | awk '{print $2}'`
}}}

When you have cleaned up the situation for all the templates, except those that are present in the workspace (because they are used in your local Quattor configuration) but don't need to be in the repository, you can do the final cleanup with :
{{{
rm -f `svn status | grep '^\?' | awk '{print $2}'`
}}}

When there is no more templates flagged as `!` (missing) or `?` (unknown), you can commit your changes to the repository. Please, take care of setting up a useful message for the commit as this is the only source for a ChangeLog...

After importing all the changes with `directory-sync` (or manually), before committing them into your SCDB, check that you can compile them successfully and that the changes in the profiles look appropriate (generally mainly some RPMS update). For this last check you can use `src/utils/profiles/compare_profile` script. After integrating the new templates, identify the changed templates, review the change if necessary and revert those not appropriate. You'll probably need to revert changes of all templates in directories `repository` (these templates are site specific but some of them are required to reside in standard directories for multiple version support).

== Fresh Installation of New Middleware Templates ===

In this scenario, you first have to download the new version of the templates, as in previous scenario. After, rather than sync'ing your templates with the release, you can copy the middleware templates provided by the release (e.g. `grid/glite-3.0.0`) in a new directory under `cfg/grid` in your CDB/SCDB and add them to the repository (use `svn add` for SCDB).

After you can select the deployment of the new release on a per cluster basis. For this, you have to edit `cluster.build.properties` of the cluster and update the include path, replacing references to previous release of middleware templates by reference to the new one you just added.


=== Controlling gLite Updates Installation ===

QWG templates releases deliver the last gLite updates available at the time of the release. There is no equivalent between QWG release number (`-n`) and gLite update numbers. Sometimes one QWG templates release deliver several gLite updates. In each QWG release, there is a default associated gLite update (generally the last one).

To allow site as much flexibility as possible in selecting what gLite update to install on a specific node, cluster or site, starting with QWG release 3.0.2-10, a release provides each gLite update separatly, not just the last one. You can select the gLite update you want to deploy by defining variable `GLITE_UPDATE_VERSION` at the beginning of your node profile, in `pro_site_cluster_info.tpl' or in gLite site parameters `pro_lcg2_config_site` respectively. The value specified must be a string and is generally the gLite update number : in fact it needs to match a directory name under [source:templates/trunk/grid/glite-3.0.0/updates cfg/grid/glite-3.0.0/updates].

For exampple, to stay with update 15 in a specific cluster, edit `site/pro_site_cluster_info.tpl` for the cluster and add :
{{{
variable GLITE_UPDATE_VERSION = '15';
}}}