Version 91 (modified by 15 years ago) (diff) | ,
---|
How to Install Quattor with SCDB
TracNav
Table of Contents
Note : the installation process described below is as generic as possible, but does also contains a set of commands that have been tested only on Scientific Linux version 4. They may need to be modified for other platforms.
Note: you may want to refer to another step-by-step installation guide containing more details for the initial steps. When in doubt, use this page as the authoritative documentation.
OS Installation
Quattor server requires a machine installed with a default server installation of any RH-based Linux distro. There is no specific requirements for the OS configuration itself. The server can be installed by any mean available at the site (CD-Rom, Kickstart, imaging...). When Quattor server is readyn it will be possible to manage the server itself with Quattor, except for OS upgrades.
Web Server Installation
The Quattor server needs to run a Web server to serve profiles, kickstart configuration files and execute the CGI script at end of installation to change PXE boot to local disk. In addition, it is recommended (but not necessary) to use this Web server for serving RPMs.
Web server installation requires nothing specific, just the configuration of a document root with enough space if you plan to serve RPMs and the configuration of CGIs. This Web server can be shared with other usages and you can use a specific virtual host instead of a dedicated server.
Apache is the recommended Web server (installation instructions here refer to Apache) and it can be installed from the OS distribution. Note that for subversion http mode, Apache version 2 or above is needed.
Apache Recommended Settings
SCDB has no strong requirement concerning Apache configuration. It generally uses 3 distinct URLs for 3 different purposes :
- Profiles : machine profiles are served by one specific URL shared
by all machines. The files there are XML files produces by the PAN
compiler when executing
ant deploy
. - Kickstart configuration files : this URL is used to store the
Kickstart configuration file for each machines. These files are
produced by
aii-shellfe --configure
. - RPM packages : RPMs are grouped in repositories, each repository has its own URL. This is a common setting to have one common parent URL for all repositories but this is not at all a requirement.
Recommended setting for these 3 areas are :
- Restrict access to profile and Kickstart configuration to IP adresses (or subnets) matching Quattor clients, as these files may contain sensitive information like encrypted passwords or MySQL passwords (cleartext).
- Configure all these areas to ignore any
index.html
file and auto-indexing. This is particularly important for RPM repositories URLs, as presence of anindex.html
will prevent SCDB tools to get the list of RPMs in the repository.
Configuration for these areas is normally done by creating a file
/etc/httpd/conf.d/quattor.conf
with directives like the following
one for each area (replace /path/to/area
by your actual directoy
name) :
<Directory /path/to/area> Options Indexes DirectoryIndex VeryUnlikelyDirectoryIndex.none AllowOverride None </Directory>
It is also better to add the following directive in our
/etc/httpd/conf.d/quattor.conf
to work around a problem in some RPM
versions:
<IfModule mod_setenvif.c> BrowserMatch "rpm/.*" nokeepalive force-response-1.0 </IfModule>
Note: if you are installing a new Apache server, don't forget to
edit DocumentRoot
in /etc/httpd/conf/httpd.conf
to reflect your
local configuration.`
Note: even though it is easily redone, it is better to backup
quattor.conf
file.
Subversion Server
There is no need for a Subversion server dedicated to Quattor. SCDB is just one repository from the Subversion point of view. If you already run a Subversion server, you can skip the installation part and go directly to the configuration part.
Subversion Installation and Configuration
There are many possible installation options for a Subversion server. The best is to install it as Apache module, anyway. There is no requirement for the Subversion server to run on a Linux machine, even if it is the installation option documented here. You can even choose to use a Subversion server outside of your site, if you think the network connection is good enough.
If you need to install a Subversion server, the easiest is to install Apache using YUM. Another option is to retrieve the RPMs for Subversion from [http://subversion.tigris.org/project_packages.html Subversion site]. Don't forget to install the Apache module which is in a separate RPM.
A typical SVN installation with YUM is:
yum install subversion mod_dav_svn
After installing, you have to configure the Subversion server. Refer to Subversion web site for details. Configuration the SVN server typically involves:
- Creation of directory which will contain the Quattor repository (this example uses
/var/svn
):mkdir -p /var/svn
- Create Subversion repository that will be used for Quattor SCDB (don't forget to backup this directory):
svnadmin create /var/svn/quattor # Repository must be owned by Apache account chown -R apache:apache /var/svn/quattor
Apache SVN module configuration (/etc/httpd/conf.d/subversion.conf
)
must be edited to configure URL used by SVN. A typical example, based
on previously created repository (adjust paths to reflect your
configuration) is:
<Location /svn> DAV svn SVNParentPath /var/svn AuthzSVNAccessFile security/svn-repositories-access AuthType Basic AuthUserFile security/passwd AuthGroupFile security/group AuthName "Grid Tutorial SVN server" # Limit write permission to list of valid users. <LimitExcept GET PROPFIND OPTIONS REPORT> # Require SSL connection for password protection. # SSLRequireSSL Require valid-user </LimitExcept> </Location>
To configure SVN authentication for SCDB repository, you need to
create one or more accounts in /etc/httpd/security/passwd
. You can
use htpasswd
or openssl passwd -apr1
to generate an encrypted
password.
You also need to define SVN ACLs in
/etc/httpd/security/svn-repositories-access
. A typical file to start
is (it assumes the account you created is called quattormgr
, if this
is a list it must be comma separated):
[groups] quattor-mgrs = quattormgr [/] * = r @quattor-mgrs = rw
Note: even though it is easily redone, it is better to backup subversion.conf
file and files in /etc/httpd/security
.
Repository configuration
For Quattor, you need to create a repository with the standard structure inside it (or inside a branch) :
trunk
: where you make the changes to your running configurationtags
: used by SCDB administration tool to do deploymentbranches
: for alternative developments
One of the possible methods to create it with an empty Quattor repository (or repository branch) is (assuming your repository is dedicated to SCDB and its URL is http://svn.example.org/Quattor):
mkdir /tmp/scdb cd /tmp/scdb mkdir trunk mkdir tags mkdir branches svn import . http://svn.example.org/Quattor --message 'Initial repository layout'
DHCP and TFTPD installation
Install DHCPD and TFTPD server from OS distribution. You can do it with the following YUM command:
yum install dhcp tftp-server
If the DHCP server is to be used for Quattor usage only, a basic DHCP configuration (/etc/dhcpd.conf
) may be:
# DHCP server configuration authoritative; allow bootp; #allow duplicates; ddns-update-style none; #omapi-port 7921; # Use a non standard port (standard = 7911) # Edit to reflect your DNS domain name and name servers (a comma-separated list is allowed) option domain-name "lal.in2p3.fr"; option domain-name-servers nfsserv.lal.in2p3.fr; option netbios-node-type 2; # Update to reflect your IP subnet subnet 134.158.72.0 netmask 255.255.255.254 { # Parameters for the installation via PXE using pxelinux filename "quattor/pxelinux.0"; #option dhcp-class-identifier "PXEClient"; option vendor-encapsulated-options 01:04:00:00:00:00:ff; # This is required on RHEL/SL/SLC/CentOS 5.X next-server your.quattor.tftp.server.domain.com; # This is now a required line in DHCP configuration. This # option gives the behavior of the previous versions. ddns-update-style ad-hoc; option routers 134.158.72.1; }
If you want to share DHCP between Quattor and non Quattor usage, it's
probably better to move the last part (subnet...
) into a separate
file, like /etc/dhcpd/quattor.conf
and replace it in the main
configuration file by:
include "/etc/dhcpd/quattor.conf";
See man dhcpd
and man dhcpd.conf
for details about DHCP server
configuration, in particular to support multiple subnets and other
advanced features.
TFTP server is run by xinetd
. In the default configuration, it is
disabled. Enable it by editing /etc/xinetd.d/tftp
, modifying
disable
parameter from yes
to no
.
Note that default location for TFTP root in AII configuration files is
/osinstall/nbp
. It must be explicitly defined if you want to use
/tftpboot
or another location.
Quattor Server
In addition to the base system installation, you need to install the following RPMs (and their dependencies) on a Quattor server where you want to use SCDB :
- Java VM > 1.5.0
- Subversion client (preferably > 1.4)
- cdb-sync
- ncm-template
- aii-server (2.4 or higher)
- ncm-lib-blockdevices (0.18.5 or 0.20)
- aii-ks
- aii-pxelinux
- ncm-ncd
- ncm-ccm
All but Java and SVN client can be download from http://quattorsrv.lal.in2p3.fr/packages/quattor/sl. Always use the last version, unless explicitly mentioned. The easiest is to use YUM after configuring a repository refering to the previous URL. This can be donne with the following command:
cat > /root/.quattor/yumroot/quattor_sl.repo <<EOF [quattor_sl] name=quattor_sl baseurl=http://quattor.web.lal.in2p3.fr/packages/quattor/sl EOF
After configuring the YUM repository, you should be able to install the required RPMs with:
yum install aii-pxelinux ncm-ncd ncm-ccm
SCDB Initialization
To start with SCDB, you first need to install a Subversion server, an open source product. The http based repository access must be used for quattor, the standalone access won't work (limitation of the build script).
You must have created a repository for Quattor and initialized its structure as explained in section Subversion Server Installation.
To create your initial SCDB with the standard templates and the associated examples, follow the following steps:
- Populate your repository with the version of SCDB and QWG templates you want to use. The easiest is to download and use check-compile.sh (use option
-h
to get the list of available options). This will download SCDB, QWG, and compile the example to ensure that everything works. To populate thecdb
subdirectory for example, with SCDB and last version from gLite 3.1 branch do:check-compile.sh -d scdb glite-3.1
- Change current directory to the working area, for example :
cd scdb
- Checkout SCDB trunk (empty) in your SCDB working directory :
svn co http://svn.example.org/Quattor/CDB/trunk .
- Configure the repository to ignore some files produced when
compiling, using the following command :
cat > /tmp/ignore <<EOF .settings build build.saved deploy .project EOF svn propset svn:ignore -F /tmp/ignore . svn ci
- Add everything to your repository with command :
svn add *
- Commit your vanilla SDCB with :
svn ci -m 'Create initial SCDB'
Site Configuration
After copying the SCDB distribution, you need to create your first
site. You can do this by copying sites/example
directory and
customizing a few templates.
RPM Repositories
To use Quattor, you need to deploy software repositories. Even if you
want to customize it later, you are probably better to start with a
configuration similar to what is provided in repository
directory of
sites/example
directory.
You can retrieve an initial directory content for each RPM repository by downloading the contents of the URL specified in comments at the beginning of each repository templates. A convenient way to do it is:
src/utils/misc/rpmUpdates /your/dir/ectory URL_found_in_template_header > /dev/null
Note: this command produces a rather verbose output, just ignore it.
You have to do it for the following repository templates:
- OS: to start, you need only to download the OS version you plan to use. Note: there are several repositories per OS versions, be sure to download all of them.
- gLite: to start download all the repositories related to the version you are planning to use, ignore others.
- All the repositories not related to an OS or gLite version
You are probably better to remove any template you think you will not use (you can always revert your change in case of mistakes).
Note on Apache configuration for RPM repositories: it is recommended to disable the use of index files on these directories iwth the following options in Apache configuration:
<Directory "/www/htdocs/packages"> Options Indexes DirectoryIndex VeryUnlikelyDirectoryIndex.none AllowOverride None </Directory>
Basic System Configuration
Basic system configuration (network parameters, DNS servers, ...) are
grouped in template pro_site_config.tpl
and pro_site_global_variables.tpl
in site
directory of
your site. Look at comments to understand what you need to modify.
Middleware Configuration
Middleware configuration is located in template
config.tpl
in site/glite
directory of your site. Look at
comments to understand what you need to modify.
Note: do not be afraid of putting wrong values in your gLite parameters, this can easily be refined later. As a general rule, keep the example values when you don't understand how to change them.
Cluster Configuration
After creating your site, you need to create your first cluster. You
can do this by copying clusters/example-3.1
directory, removing all profiles from clusters/example-3.1/profiles
and customizing a
few templates.
Hardware description
You need to create a template describing the hardware configuration of
your machines. This is generally placed in hardware
sub-directory of
site directory. Look at examples, copy one with a configuration close to yours as a starting point. The name of the hardware template is at your convenience and will be associated with a host name later.
Note: recommendation is to have name of templates describing HW boxes derived from box location rather than its current host name as the same HW may be affected to another host later.
Adding Machine to site/databases.tpl
Before being able to configure the machine, you need to create an entry for the machine name in both variables of databases.tpl. First variable defines the address associated with the machine name, second variable defines the hardware template associated with the machine.
In both variable, the key is the escaped host name. The value is an IP address in the first case, the name of the template (relative to your site name in sites
directory) you created in the previous step for the second variable.
Creating Machine Profile
Copy an existing profile in examples corresponding to the machine type you want to create.
Note: as a first node, it is recommended to configure a site BDII as this is the most simple node. When successful with site BDII, the next ones are generally CE and one WN.
Quattor Server Final configuration
Before being able to deploy the created configuration, there is a last configuration step needed to allow deployment of the configuration after successful compilation. This involves :
- Adding a hook script to the Subversion repository to trigger the deployment
- Adding a script on the Quattor server that will be launched by the hook script, using ssh
- Configuring SSH keys to allow execution of the previous script as root (preferably) from the Apache account
- Add a CGI script on Quattor server used at end of installation of a machine to allow next boot from local disk.
- Configuration of AII
Installation of hook script and server scripts
The hook script, post-commit, is provided as part of SCDB, in the src/hooks
directory. It must be installed on your Subversion server, in the hook
directory of the repository, and given executable permission for Apache user. This script requires a configuration file /etc/quattor-deploy.conf
, see SCDB server-side customizations for details.
The other script, build-tag.pl, also provided as part of SCDB, in the src/hooks
directory, must be installed (root executable) in /root/quattor/scripts
on the Quattor server. It also requires a configuration file, /etc/build-tag.conf
. See SCDB server-side customizations for details.
build-tag.pl
requires file quattor.build.properties
to be created in the parent of the directory specified in svn_cache
paramater of build-tag.pl configuration file or its default value (/root/quattor
for example provided here). A template of this file is available in SCDB distribution, in src/hooks
directory. It must be edited to reflect your local configuration.
Note: you can download the last version of these scripts from QWG repository with the following command:
wget --no-check-certificate "https://svn.lal.in2p3.fr/LCG/QWG/SCDB/trunk/src/hooks/post-commit" -O /var/svn/quattor/hooks/post-commit chmod 755 /var/svn/quattor/hooks/post-commit wget --no-check-certificate "https://svn.lal.in2p3.fr/LCG/QWG/SCDB/trunk/src/hooks/build-tag.pl" -O /root/quattor/scripts/build-tag.pl chmod 755 /root/quattor/scripts/build-tag.pl
For more details about these scripts and their customization, see the page on SCDB server-side customizations.
Creation of SSH Keys
Currently, deployment of new version of the templates is done by the hook script triggered by ant deploy
executing the server script build-tag.pl
through ssh. There is no way to enter a password at this time, thus ssh must be configured in such a way that the Apache account on the Subversion server can do a ssh connection as root on the Quattor server, without password. The easiest is to use ssh keys to do that.
Note: if you run SVN server on the Quattor server, an alternative to SSH is to use sudo
. This currently requires to use a specific variant of the post-commit
script, post-commit.sudo. After installing the script, instead of configuring SSH keys, you need to add the following configuration lines (customize them to reflect your local configuration) to sudo
with visudo
utility:
Cmnd_Alias QUATTORDEPLOY=/root/quattor/scripts/build-tag.pl * apache ALL = NOPASSWD: QUATTORDEPLOY Defaults:apache !requiretty
The last line is required on later versions of RHEL-like operating systems. If it is not given, the sudo will silently fail.
Post-installation CGI Script
At the end of a machine installation, as part of the Kickstart
post-intallation script, a CGI script is executed on the Quattor
server to change PXE configuration in order for the machine to boot
from local disk next time. This allows to set PXE as the first boot
device in the BIOS and control re-installation via aii-shellfe
command.
This script, aii-installack.cgi
, can be found in SCDB directory
src/cgis
. It must be placed on the Web server running on the Quattor
server, in the directory for CGIs.
The apache server must be able to run that script as root. Best is to
have sudo
installed and use visudo
to add the following to
/etc/sudoers
:
Cmnd_Alias AIIACKCGI=/usr/sbin/aii-shellfe apache ALL = NOPASSWD: AIIACKCGI
Configuration of AII
This involves 2 separate steps :
- Customization of
/etc/aii/aii*.conf
files - Customization of AII-related variables in templates
There are 2 AII configuration files that need to be customized to reflect your site configuration:
/etc/aii/aii-shellfe.conf
: a typical file is as follows, with paths edited to reflect your configuration.# URL corresponding to compiled profiles generated by ant deploy cdburl=http://quattor.web.lal.in2p3.fr/profiles # use_fqdn must be set to true use_fqdn=true # Directory where to write Kickstart configuration files produced by aii-shellfe. # Must match the directory served by URL defined in template variable QUATTOR_PROFILE_URL. osinstalldir = /www/htdocs/ks # Directory where pxelinux.cfg is installed. Default is the recommended locaiton. nbpdir = /tftpboot/quattor/pxelinux.cfg
/etc/aii/aii-dhcp.conf
: check thatdhcpconf
andrestartcmd
command match your configuration and edit as necessary. The file referred bydhcpconf
must be writable from the AII server.restartcmd
may launch a command on a remote machine through appropriate means.
To customize AII configuration files, located in /etc
and named
aii-*.conf
, refer to the comment in each files. Main parameters to
customize are in
aii-shellfe.conf
:
cdburl
: URL to use to download profilesosinstalldir
: directory where to place kickstart configuration files produced by AII
Note: if /etc/aii
doesn't exist or is empty on your server, copy templates of these files located in /usr/share/doc/aii-x.y.z/eg
(with x.y.z
matching AII version installed).
There are a few variables to customize in site templates to reflect your Quattor and AII configuration, mainly :
QUATTOR_PROFILE_URL
: URL to use to download machine profiles.AII_OSINSTALL_SRV
: Name of the Web server serving kickstart- configuration files and RPMs.
AII_ACKSRV
: Name of the Web server to use for the post-installation CGI. Defaults toAII_OSINSTALL_SRV
AII_ACKCGI
: post-installation CGI URL. Defaults to/cgi-bin/aii-installack.cgi
.
These variables are generally defined site-wide, in the template
pro_site_global_variables.tpl
located in site directory. Look at
provided examples, in SCDB distribution.
Downloading the distribution's images
If you want to perform network-based installations, you need to download the distribution's CDs. They contain the kernel and initrd to be used during the installation, which will re-direct to Red Hat's installer. This installer is also located on the CD (usually the first CD) or DVD of your distribution.
The easiest way is to download the full DVD of your distro, f.i, SL:
wget http://.../distro-version.iso
Then, mount it somewhere Apache can read to. For instance,
/var/www/html/your_platform
:
mount -o bind /path/to/dvd/image /var/www/html/sl520-x86_64
Add it to your fstab, if needed. Next, you'll need to copy the files used for PXE somewhere the TFTP server can reach them. Their location depends on the distribution, it's usually on a directory called pxeboot:
mkdir /osinstall/nbp/<platform> cp /var/www/html/<platform>/.../pxeboot/* /osinstall/nbp/<platform>
Compiling and Deploying
After the configuration is finished, you can try to compile your first profile, deploy it and install the machine. This involves the following steps :
- In SCDB (working area copy) top level directory :
- Update of RPM repository templates :
external/ant/bin/ant update.rep.templates
- Profile compilation and deployment (deployment will not occur until compilation succeds) :
external/ant/bin/ant deploy
- Update of RPM repository templates :
- On the Quattor server :
- Creation of Kickstart configuration file for the machine :
aii-shellfe --configure your.machine.domain
- Update of DHCP and PXE for the machine to be installed at next boot :
aii-shellfe --install your.machine.domain
- Creation of Kickstart configuration file for the machine :
Note: you can ignore the warning returned by aii-shellfe
about base_url is not defined in configuration
.
Troubleshooting Initial Installation
Deployment doesn't work
Look at SCDB server-side customizations page.