SCDB Server Configuration
TracNav
Table of Contents
This page describes the server side of SCDB, made of the SVN server and a Quattor deployment server, the specific components used and how to customize their configuration. It doesn't explain how to install the SVN server and the Quattor server and how to configure them for SCDB.
Server-side components of SCDB consist in 2 scripts involved in configuration deployment and triggered by ant deploy
:
- SVN post-commit hook script
- Deployment script
Both scripts are distributed as part of SCDB tools, inside src/hooks directory. See the the installation documentation for more information.
Subversion Hook Script
This script is trigger by ant deploy
when creating the deployment tag. Its main role is to do several checks and if necessary to launch the deployment script.
This script must be installed on the Subversion server (not necessarily the Quattor server), inside the hooks
directory of the SCDB repository (the repository itself, not a working copy). For this you need write access to the directory containing the repository on the Subversion server.
This script must be named post-commit
and be executable by the account the SVN server is running under (typically the Apache account). The source file for this script is post-commit.py.
Note: an obsolete version of this script was called post-commit
(and is a shell script). All the information here doesn't apply to this obsolete version whose configuration file had a different format, despite the same name. When migrating from the obsolete version to the new one, the configuration file must be replaced.
This script requires a configuration file, /etc/quattor-deploy.conf
(common with the deployment script when running on the same machine). All the configuration options supported and their default values are defined at the beginning of the script. The configuration file has several sections. One of them, [scdb]
, is common to the hook script and the deployment script. The main options available and their default values are:
[post-commit] # Script launched by the script to actually do the deployment deploy_script : /root/quattor/scripts/build-tag.py # Name of the deployment server where to run the deploy_script. Used only with ssh. # This can be a space-separated list (not yet implemented, see https://trac.lal.in2p3.fr/LCGQWG/ticket/46). #deploy_server : quattorsrv.example.org # Userid to use to run deploy_script deploy_user : root # notify_xxx are used to configure email notification in case of errors. # If notif_from or notif_to is undefined, email notification is disabled notif_mailer : localhost #notif_from=Quattor Deployment <noreply@lal.in2p3.fr> #notif_to=jouvin@lal.in2p3.fr notif_subject_prefix : [Quattor-Deploy] notif_subject : Failed to deploy revision %s of SCDB configuration # Default should be appropriate. Set to false if your client doesn't handle properly returned output. # When set to false, no message is printed on stdout, except if verbose is > 0. report_error_to_svn: yes # When false, ssh is used instead. This requires deploy_server to be defined too. use_sudo : yes # Log operations in /tmp/quattor-post-commit.log verbose: 0 [ssh] cmd: /usr/bin/ssh options: -o PasswordAuthentication=no [sudo] cmd: /usr/bin/sudo options: -H [scdb] # URL associated with the repository root #repository_url: http://svn.example.com/scdb # Branch where to create SCDB deployment tags # Keep consistent with quattor.build.properties if not using default values. tags_branch: /tags # Branch corresponding to SCDB trunk (only branch allowed to deploy) # Keep consistent with quattor.build.properties if not using default values. trunk_branch: /trunk
The only required options for which no default value is provided is repository_url
in [scdb]
section. It must match the root URL of your SVN repository (as returned by svn info
).
Note on using sudo
: recent versions of sudo
disable the use of sudo
without a tty by default. This must be change in order to use sudo
in the context of the deployement script. To to this, run visudo
and comment out line Defaults requiretty
.
Deployment Script
This script is launched by the hook script and does the real work by updating local working copy of the SCDB repository to the tag to deploy and by calling appropriate ant
tasks.
It needs to be installed on the Quattor server and will be called through sudo or ssh by the hook script (you need to configure a SSH key without password between your Subversion server and your Quattor server to use ssh). The default location to install the script is /root/quattor/scripts
but you can put it wherever you want as soon as you update the hook script configuration ([post-commit]
section) accordingly. The source of this script is build-tag.py.
Note: an obsolete version of this script was called build-tag.pl
(written in Perl). All the information here doesn't apply to this obsolete version whose configuration file had a different format. When migrating from the obsolete version to the new one, the configuration file must be replaced.
This script requires a configuration file, /etc/quattor-deploy.conf
(common with the hook script when running on the same machine). All the configuration options supported and their default values are defined at the beginning of the script. The configuration file has several sections. One of them, [scdb]
, is common to the hook script and the deployment script. The main options available and their default values are:
[build-tag] # If not starting with /, relative to directory specified by option svn_cache ant_cmd: external/ant/bin/ant # ant options (passed through env variable ANT_OPTS) #ant_opts: -Xmx2048M # ant stdout: allow to redirect ant output to a file for debugging purposes (default is PIPE) # Default changed to a file because of problem in subprocess module if the # output is very large (Python 2.4) leading to the parent/child communication # to hang. ant_stdout: /tmp/ant-deploy-notify.log #ant_stdout: PIPE # ant target to do the deployment. Default should be appropriate. ant_target: deploy.and.notify # If not starting with /, relative to /usr/java. java_version: latest # If not starting with /, relative to parent of this directory script svn_cache: svncache # Number of retries for SVN switch to new tag in case of error switch_retry_count: 1 # Verbosity level verbose: 0 [scdb] # URL associated with the repository root. Required parameter without default value. #repository_url: http://svn.example.com/scdb # Branch where to create SCDB deployment tags # Keep consistent with quattor.build.properties if not using default values. tags_branch: /tags # Branch corresponding to SCDB trunk (only branch allowed to deploy) # Keep consistent with quattor.build.properties if not using default values. trunk_branch: /trunk
The only required options for which no default value is provided is repository_url
in [scdb]
section. It must match the root URL of your SVN repository (as returned by svn info
).
This script needs to find a file quattor.build.properties in the parent directory of SCDB local cache (by default, parent directory for this script location).
Troubleshooting Server-side Scripts
If there is a problem during deployment, after entering command ant deploy
, people listed in notify_to
configuration option of the hook script should receive an email describing the problem. If they don't receive an email about the deployment failure, check your SMTP configuration and your notification parameters.
Note : notification on deployment failures is optional and errors should be reported through the ant
command since SCDB 2.3.0.
To troubleshoot server side configuration problems, it is necessary to execute manually the server scripts. This is a two-step processes :
- Check successful execution of deployment script,
build-tag.py
. - Check successful execution of SVN hook script.
Troubleshooting deployment script
To troubleshoot deployment script, build-tag.py
, you must log on the Quattor server as the user configured to run deployment script (generally root
).
The step-by-step procedure to troubleshoot deployment script (examples below assume it is in default location /root/quattor/scripts
) is :
- Look in
/etc/quattor-deploy.conf
the location of SVN cache on the Quattor server. If it is not defined, the default location is/root/quattor/svncache
. In the following steps, replace this default location by whatever is appropriate at your site. - Check the last tag checked out with the following command :
svn info /root/quattor/svncache ---> look at the current tag name, everything after /tags/ in the URL (without leading /)
- Check the tag has been successfully checked out. If the following command updates anything, it means the previous attempt to switch to this tag was unsuccessful. If the command fails again, retry it until it is successful. Failure to switch to a tag is generally related to a Subversion server problem, not to Quattor itself : check Subversion server logs for more information.
svn update
- Try to redeploy the same tag, enabling verbose output, with the following command :
/root/quattor/scripts/build-tag.py --debug TAG-VALUE
If SVN cache is empty (not recognized as a valid SVN working copy by svn info
), you need to identify last tag in tags
branch of Quattor SCDB repository using svn ls
.
Note: if you are still using the obsolete deployment script, build-tag.pl
, most of the troubleshooting steps should apply but you need to adapt the configuration file and script names.
Troubleshooting SVN hook script
Most of the deployment problems, after initial configuration, are related to Deployment script. As the hook script mainly launches the deployment script, it is important to complete previous checks before troubleshooting SVN hook script.
The main cause of problems in the hook scripts is related to launching the deployment script from the hook script. The reason for the problems and the troubleshooting steps are different whether you use sudo
or ssh
to run the deployment script on the Quattor deployment server.
- When using
ssh
, the main reason is invalid SSH keys for communication between SVN server and the Quattor server and invalid configuration. To troubleshoot this, log in to the Subversion server as the same user as your SVN server (generally Apache account) and try to ssh to the Quattor deployment server using the same account as the one used by the deployment script (typicallyroot
). - When using
sudo
, log in to the Subversion server as the same user as your SVN server (generally Apache account) and try to sudo under the deployment account (the one used to run the deployment script, typicallyroot
). One well-known issue with recent version ofsudo
is that you need to enable the use ofsudo
by processes without a tty (this must be done withvisudo
).
When you assessed the ability to launch the deployment script is working properly, try to execute the hook script under the SVN server account. This involves the following steps:
- Log on Subversion server as the same user as your Subversion server (generally Apache account).
- Retrieve the repository revision associated with the last deployment tag. For this, you need to run the following command (with
tag_url
the URL associated with the tag branch of the Quattor repository):svn log --limit 1 tag_url
- Go to the
hooks
directory of Subversion SCDB repository and run the following command (REPOSITORY_PATH
is the filesystem path to the repository) :./post-commit --debug REPOSITY_PATH REVISION
Archived Documentation
The information in this section is kept here for reference. It documents the configuration file format of the obsolete hook script and deployment script.
Hook script (obsolete)
This script requires a configuration file to be present, /etc/quattor-deploy.conf
. The configuration file must contain only shell variable definition (sh
syntax). The required variables are :
# Quattor server name QUATTORSRV="quattorsrv.lal.in2p3.fr" # Recipients of status mail and location for checkout. NOTIFY="quattor-mgrs@lal.in2p3.fr"
Other supported variables are :
# Script on Quattor server to deploy new configuration # Default : DEPLOYSCRIPT=/root/quattor/scripts/build-tag.pl # User to run the deployment script # Default : DEPLOYUSER=root # Full paths to commands. # Defaults are appropriate for RHEL/SL Linux. # MAILER="/bin/mail" # SVNLOOK="/usr/bin/svnlook" # SSH="/usr/bin/ssh" # Use sudo rather than ssh between SVN server and quattor deployment server # Default: SSH # USE_SUDO=1 # Default prefix for mail subject in case of error # Default : ERROR_PREFIX='[Quattor-Deploy]'
Deployment Script (obsolete)
/etc/build-tag.conf
allows to customize build-tag.pl according to local needs. It must be present. This file must contain key/value pairs (with '=' between key and value, keys are not case sensitive). Supported configuration options are :
debug
: display additional information if non zero.java_home
: path of Java virtual machine.java_version
: alternative tojava_home
. Must match a directory under/usr/java
. Default : $JAVA_HOME if defined.svn_cache
: location to use for SCDB repository local cache. Default :svncache
at same level as the directory where the script is installed.use_svn
: if non zero, force the use ofsvn
command even ifjsvn
exists. Default : usejsvn
fromsvn_cache
if it is present. Note : usingjsvn
may affect performances.tags_svn_url
(required) : SVN URL for SCDB repositorytags
branch
An /etc/build-tag.conf
example is :
use_svn = 1 java_version = jdk1.6.0 tags_svn_url = http://grid280.lal.in2p3.fr/svn/quattor/tags