= How to Install Quattor with SCDB =
[[TracNav]]

[[TOC(inline)]]

== Preliminary note ==

The installation process described below is as generic as possible, but does also contains a set of commands that have been tested only on Scientific Linux version 3. They may need to be modified for other plateforms.

== Web Server Installation ==

The Quattor server needs to run a Web server to serve profiles, kickstart configuration files and execute the CGI script at end of installation to change PXE boot to local disk. In addition, it is recommended (but not necessary) to use this Web server for serving RPMs.

Web server installation requires nothing specific, just the configuration of a document root with enough space if you plan to serve RPMs and the configuration of CGIs. This Web server can be shared with other usages and you can use a specific virtual host instead of a dedicated server.

Apache is the recommended Web server (installation instructions here refer to Apache) and it can be installed from the OS distribution. Note that for subversion http mode, Apache version 2 or above is needed.


== Subversion Server ==

There is no need for a Subversion server dedicated to Quattor. SCDB is just one repository from the Subversion point of view. If you already run a Subversion server, you can skip the installation part and go directly to the configuration part.

=== Subversion installation ===

There are many possible installation options for a Subversion server. The best is to install it as Apache module, anyway. There is no requirement for the Subversion server to run on a Linux machine, even if it is the installation option documented here. You can even choose to use a Subversion server outside of your site, if you think the network connection is good enough.

If you need to install a Subversion server, the easiest is to install Apache and then retrieve the RPMs for Subversion from [http://subversion.tigris.org/project_packages.html Subversion site]. Don't forget to install the Apache module which is in a separate RPM.

After installing, you have to configure the Subversion server. Look at Quattor installation guide on [http://quattor.org Quattor] web site.

Quick setup:

{{{
# Create SVN repository
mkdir -p /var/svn/quattor
svnadmin create /var/svn/quattor
# Fetch the quattor specific post-commit hook
wget  --no-check-certificate "https://trac.lal.in2p3.fr/Quattor/browser/trunk/src/hooks/post-commit?format=raw" -O /var/svn/quattor/hooks/post-commit
# Quattor deployment scripts (used by post-commit svn hook)
mkdir -p /root/quattor/scripts
wget  --no-check-certificate "https://trac.lal.in2p3.fr/Quattor/browser/trunk/src/hooks/build-tag.pl?format=raw" -O /root/quattor/scripts/build-tag.pl

}}}

The {{{post-commit}}} file have to be edited. In particular, check for:
{{{QUATTORSRV}}}, {{{DEPLOYSCRIPT}}}, {{{MAILER/SVNLOOK/SSH}}} and {{{NOTIFY}}} variables (everything should be self-explaining). ''Note to Michel: toutes les variables à modifier devraient être groupées au début du fichier.'' Also in {{{post-commit}}}, beware of the regular expression used to match a tag: it may be wrong for you, tune it as needed.

One should also edit {{{build-tag.pl}}} file, look for {{{JAVA_HOME}}}, {{{$repo}}} and {{{$cache}}}.

The SVN repository have to be accessed by {{{http(s):}}} method (not {{{file:}}}), this imply installation and configuration of {{{mod_dav_svn}}} for Apache.

The SVN repository should belong to the identity that run the web server:

{{{
chown -R apache:apache /var/svn/quattor
}}}

=== Apache for subversion and package service ===

Files  {{{/etc/httpd/conf.d/subversion.conf}}} and  {{{/etc/httpd/conf.d/ssl.conf}}} (if using ssl) have to be edited and configured.

In {{{/etc/http/conf/httpd.conf}}}, you may have to add the following lines to avoid a bug in the http interface of the rpm client.

{{{
<IfModule mod_setenvif.c>
  BrowserMatch "rpm/.*" nokeepalive force-response-1.0
</IfModule>
}}}

=== Repository configuration ===

For Quattor, you need to create a repository with the standard structure inside it (or inside a branch) :
 * `trunk` : where you make the changes to your running configuration
 * `tags` : used by SCDB administration tool to do deployment
 * `branches` : for alternative developments

For example:

{{{
mkdir toto
cd toto
mkdir scdb
mkdir scdb/trunk
mkdir scdb/tags
mkdir scdb/branches
svn import . https://svn.server.tld/svn/quattor --message 'Initial repository layout'
}}}

Then, initial checkout will be:

{{{
svn checkout https://svn.server.tld/svn/quattor/scdb
}}}

== DHCP and TFTPD installation ==

Install DHCPD and TFTPD server from OS distribution. Do a basic configuration of DHCPD server (name servers...).

Template for dhcp configuration can be found in aii-server package, under {{{/usr/share/doc/aii-1.0.44/eg/dhcpd.conf}}}.

Template for tftpd configuration (by xinetd at {{{/etc/xinetd.d/tftp}}}) can be found in aii-server package, under {{{/usr/share/doc/aii-1.0.44/eg/tftp.example}}}.

Note that prefered location for quattor tftp installation is {{{/osinstall/nbp}}}, but it should work under {{{/tftpboot}}} as well.


== Quattor Server ==

In addition to the base system installation, you need to install the following RPMs on a Quattor server where you want to use SCDB :

 * Java VM > 1.5.0
 * Subversion client
 * PAN compiler
 * cdb-sync
 * ncm-template
 * aii-server

All but Java and SVN client can be download from http://quattorsrv.lal.in2p3.fr/packages/quattor/sl-i386. Always use the last version, unless explicitly mentionned.

== SCDB Initialization ==

To start with SCDB, you first need to install a [http://subversion.tigris.org Subversion] server, an open source product. The http based repository access '''must''' be used for quattor, the standalone access wont work (limitation of the build script).

After you have a Subversion server installed, you need to :
 * Create a repository or a branch in an existing repository where SCDB will be stored
 * Checkout this repository/branch (`svn co`)
 * Import the basic SCDB structure from [source:SCDB].
 * Import QWG templates in `cfg` directory, according to the version you need. See QWG [wiki:Download download] instructions.

A tool originally written for QWG templates developpers, [source:templates/trunk/tools/check-compile.sh check-compile.sh], simplifies the two last steps (import of standard templates), executing both operations and compiling examples. Use option `-h` to get the list of available options. A typical use of this tool (replace the release tag `gLite-3.0.2-2` by the QWG templates release you want to download) is :
{{{
check-compile.sh -d your_scdb_dir /templates/tags/gLite-3.0.2-2
}}}


== Site Configuration ==

After copying the SCDB distribution, you need to create your first site. You can do this by copying `sites/example` directory and customizing a few templates.

=== RPM Repositories ===

To use Quattor, you need to deploy software repositories. Even if you want to customize it later, you are probably better to start with a configuration similar to what is provided in `repository` directory of `sites/example` directory. You can retrieve an initial directory content  for each RPM repository by downloading the contents of the URL specified in comments at the beginning of each repository templates.

=== Basic System Configuration ===

Basic system configuration (network parameters, DNS servers, ...) are grouped in template `pro_site_cluster_info.tpl` in `site` directory of your site. Look at comments to understand what you need to modify.

=== Middleware Configuration ===

Middleware configuration is located in template `pro_lcg2_config_site.tpl` in `site` directory of your site. Look at comments to understand what you need to modify.


== Cluster Configuration ==

After creating your site, you need to create your first cluster. You can do this by copying `clusters/example` directory and customizing a few templates.

=== Hardware description ===

You need to create a template describing the hardware configuration of your machines. This is generally placed in `hardware`sub-directory of site directory. Look at examples.

=== Adding Machine to pro_site_database.tpl ===

Before being able to configure the machine, you need to create an entry for the machine name in both tables of  `pro_site_database.tpl`. First entry defines the address associated with the machine name, second entry defines the hardware template associated with the machne.

=== Creating Machine Profile ===

Copy an existing profile in examples corresponding to the machine type you want to create.

== Quattor Server Final configuration ==

Before being able to deploy the created configuration, there is a last configuration step needed to allow deployment of the configuration after successful compilation. This involves :

 * Adding a hook script to the Subversion repository to trigger the deployment
 * Adding a script on the Quattor server that will be launched by the hook script, using ssh
 * Configuring SSH keys to allow execution of the previous script as root (preferably) from the Apache account
 * Add a CGI script on Quattor server used at end of installation of a machine to allow next boot from local disk.
 * Configuration of AII


=== Installation of hook script and server script ===

The hook script, `post-commit`, is provided as part of SCDB, in the `src/hooks` directory. It must be installed on your Subversion server, in the `hook` directory of the repository, and given executable permission for Apache user.

The other script, `build-tag.pl`, also provided as part of SCDB, in the `src/hooks` directory, must be installed (root executable) in `/root/quattor/scripts` on the Quattor server.

`build-tag.pl` requires file `quattor.build.properties` to be created in `/root/quattor`. A template of this file is available in SCDB distribution, in `src/hooks` directory. It must be edited to reflect your local configuration.


=== Creation of Ssh Keys ===

Currently, deployment of new version of the templates is done by the hook script triggered by `ant deploy` executing the server script `build-tag.pl` through ssh. There is no way to enter a password at this time, thus ssh must be configured in such a way that the Apache account on the Subversion server can do a ssh connection as root on the Quattor server, without password. The easiest is to use ssh keys to do that.


=== Post-installation CGI Script ===

At the end of a machine installation, as part of the Kickstart post-intallation script, a CGI script is executed on the Quattor server to change PXE configuration in order for the machine to boot from local disk next time. This allows to set PXE as the first boot device in the BIOS and control re-installation via `aii-shellfe` command.

This script, `build-tag.pl`, can be found in SCDB directory `src/cgis`. It must be placed on the Web server running on the Quattor server, in the directory for CGIs.

The apache server must be able to run that script as root. Best is to have {{{sudo}}} installed and use {{{visudo}}} to add the following to {{{/etc/sudoers}}}:
 
{{{apache sbgat419.in2p3.fr=(ALL) NOPASSWD: /usr/sbin/aii-shellfe}}}


=== Configuration of AII ===

This involves 2 separate steps :
 * Customization of `/etc/aii*.conf` files
 * Customization of AII related variables in templates

To customize AII configuration files, located in `/etc` and named `aii-*.conf`, refer to the comment in each files. Main parameters to customize are the URL to use to download profiles (in `aii-shellfe.conf`) and the directory where to place kickstart configuration files produced by AII (in `aii-osinstall.conf`).

There are a few variables to customize in site templates to reflect your Quattor and AII configuration, mainly :

 * `QUATTOR_PROFILE_URL` : URL to use to download machine profiles.
 * `AII_OSINSTALL_SRV` : Name of the Web server serving kickstart configuration files and RPMs.
 * `AII_ACKSRV` : Name of the Web server to use for the post-installation CGI. Defaults to `AII_OSINSTALL_SRV`
 * `AII_ACKCGI` : post-installation CGI URL. Defaults to `/cgi-bin/aii-installack.cgi`.
 * `AII_OSINSTALL_TEMPLATE` : name of the Kickstart configuration template to use. Defaults to `i386_sl3_ks.conf`.

These variables are generally defined site-wide, in the template `pro_site_global_variables.tpl` located in site directory. Look at provided examples, in SCDB distribution.


== Compiling and Deploying ==

After the configuration is finished, you can try to compile your first profile, deploy it and install the machine. This involves the following steps :

 * In SCDB (working area copy) top level directory :
   * Update of RPM repository templates :
{{{
external/ant/bin/ant update.rep.templates
}}}
   * Profile compilation and deployment (deployment will not occur until compilation succeds) :
{{{
external/ant/bin/ant deploy
}}} 
 * On the Quattor server :
   * Creation of Kickstart configuration file for the machine :
{{{
aii-shellfe --configure your.machine.domain
}}}
   * Update of DHCP and PXE for the machine to be installed at next boot :
{{{
aii-shellfe --install your.machine.domain
}}}