wiki:Doc/SCDB/Server

SCDB Server Configuration

This page describes the server side of SCDB, made of the SVN server and a Quattor deployment server, the specific components used and how to customize their configuration. It doesn't explain how to install the SVN server and the Quattor server and how to configure them for SCDB.

Server-side components of SCDB consist in 2 scripts involved in configuration deployment and triggered by ant deploy:

Both scripts are distributed as part of SCDB tools, inside src/hooks directory. See the the installation documentation for more information.

Subversion Hook Script

This script is trigger by ant deploy when creating the deployment tag. Its main role is to do several checks and if necessary to launch the deployment script.

This script must be installed on the Subversion server (not necessarily the Quattor server), inside the hooks directory of the SCDB repository (the repository itself, not a working copy). For this you need write access to the directory containing the repository on the Subversion server.

This script must be named post-commit and be executable by the account the SVN server is running under (typically the Apache account). The source file for this script is post-commit.py.

Note: an obsolete version of this script was called post-commit (and is a shell script). All the information here doesn't apply to this obsolete version whose configuration file had a different format, despite the same name. When migrating from the obsolete version to the new one, the configuration file must be replaced.

This script requires a configuration file, /etc/quattor-deploy.conf (common with the deployment script when running on the same machine). All the configuration options supported and their default values are defined at the beginning of the script. The configuration file has several sections. One of them, [scdb], is common to the hook script and the deployment script. The main options available and their default values are:

[post-commit]
# Script launched by the script to actually do the deployment
deploy_script : /root/quattor/scripts/build-tag.py
# Name of the deployment server where to run the deploy_script. Used only with ssh.
# This can be a space-separated list (not yet implemented, see https://trac.lal.in2p3.fr/LCGQWG/ticket/46).
#deploy_server : quattorsrv.example.org
# Userid to use to run deploy_script
deploy_user : root
# notify_xxx are used to configure email notification in case of errors.
# If notif_from or notif_to is undefined, email notification is disabled
notif_mailer : localhost
#notif_from=Quattor Deployment <noreply@lal.in2p3.fr>
#notif_to=jouvin@lal.in2p3.fr
notif_subject_prefix : [Quattor-Deploy]
notif_subject : Failed to deploy revision %s of SCDB configuration
# Default should be appropriate. Set to false if your client doesn't handle properly returned output.
# When set to false, no message is printed on stdout, except if verbose is > 0.
report_error_to_svn: yes
# When false, ssh is used instead. This requires deploy_server to be defined too.
use_sudo : yes
# Log operations in /tmp/quattor-post-commit.log
verbose: 0

[ssh]
cmd: /usr/bin/ssh
options: -o PasswordAuthentication=no

[sudo]
cmd: /usr/bin/sudo
options: -H

[scdb]
# URL associated with the repository root
#repository_url: http://svn.example.com/scdb
# Branch where to create SCDB deployment tags
# Keep consistent with quattor.build.properties if not using default values.
tags_branch: /tags
# Branch corresponding to SCDB trunk (only branch allowed to deploy)
# Keep consistent with quattor.build.properties if not using default values.
trunk_branch: /trunk

The only required options for which no default value is provided is repository_url in [scdb] section. It must match the root URL of your SVN repository (as returned by svn info).

Note on using sudo: recent versions of sudo disable the use of sudo without a tty by default. This must be change in order to use sudo in the context of the deployement script. To to this, run visudo and comment out line Defaults requiretty.

Deployment Script

This script is launched by the hook script and does the real work by updating local working copy of the SCDB repository to the tag to deploy and by calling appropriate ant tasks.

It needs to be installed on the Quattor server and will be called through sudo or ssh by the hook script (you need to configure a SSH key without password between your Subversion server and your Quattor server to use ssh). The default location to install the script is /root/quattor/scripts but you can put it wherever you want as soon as you update the hook script configuration ([post-commit] section) accordingly. The source of this script is build-tag.py.

Note: an obsolete version of this script was called build-tag.pl (written in Perl). All the information here doesn't apply to this obsolete version whose configuration file had a different format. When migrating from the obsolete version to the new one, the configuration file must be replaced.

This script requires a configuration file, /etc/quattor-deploy.conf (common with the hook script when running on the same machine). All the configuration options supported and their default values are defined at the beginning of the script. The configuration file has several sections. One of them, [scdb], is common to the hook script and the deployment script. The main options available and their default values are:

[build-tag]
# If not starting with /, relative to directory specified by option svn_cache
ant_cmd: external/ant/bin/ant
# ant options (passed through env variable ANT_OPTS)
#ant_opts: -Xmx2048M
# ant stdout: allow to redirect ant output to a file for debugging purposes (default is PIPE)
# Default changed to a file because of problem in subprocess module if the 
# output is very large (Python 2.4) leading to the parent/child communication
# to hang.
ant_stdout: /tmp/ant-deploy-notify.log
#ant_stdout: PIPE
# ant target to do the deployment. Default should be appropriate.
ant_target: deploy.and.notify
# If not starting with /, relative to /usr/java.
java_version: latest
# If not starting with /, relative to parent of this directory script
svn_cache: svncache
# Number of retries for SVN switch to new tag in case of error
switch_retry_count: 1
# Verbosity level
verbose: 0

[scdb]
# URL associated with the repository root. Required parameter without default value.
#repository_url: http://svn.example.com/scdb
# Branch where to create SCDB deployment tags
# Keep consistent with quattor.build.properties if not using default values.
tags_branch: /tags
# Branch corresponding to SCDB trunk (only branch allowed to deploy)
# Keep consistent with quattor.build.properties if not using default values.
trunk_branch: /trunk

The only required options for which no default value is provided is repository_url in [scdb] section. It must match the root URL of your SVN repository (as returned by svn info).

This script needs to find a file quattor.build.properties in the parent directory of SCDB local cache (by default, parent directory for this script location).

Troubleshooting Server-side Scripts

If there is a problem during deployment, after entering command ant deploy, people listed in notify_to configuration option of the hook script should receive an email describing the problem. If they don't receive an email about the deployment failure, check your SMTP configuration and your notification parameters.

Note : notification on deployment failures is optional and errors should be reported through the ant command since SCDB 2.3.0.

To troubleshoot server side configuration problems, it is necessary to execute manually the server scripts. This is a two-step processes :

  1. Check successful execution of deployment script, build-tag.py.
  2. Check successful execution of SVN hook script.

Troubleshooting deployment script

To troubleshoot deployment script, build-tag.py, you must log on the Quattor server as the user configured to run deployment script (generally root).

The step-by-step procedure to troubleshoot deployment script (examples below assume it is in default location /root/quattor/scripts) is :

  1. Look in /etc/quattor-deploy.conf the location of SVN cache on the Quattor server. If it is not defined, the default location is /root/quattor/svncache. In the following steps, replace this default location by whatever is appropriate at your site.
  2. Check the last tag checked out with the following command :
    svn info /root/quattor/svncache
    ---> look at the current tag name, everything after /tags/ in the URL (without leading /)
    
  3. Check the tag has been successfully checked out. If the following command updates anything, it means the previous attempt to switch to this tag was unsuccessful. If the command fails again, retry it until it is successful. Failure to switch to a tag is generally related to a Subversion server problem, not to Quattor itself : check Subversion server logs for more information.
    svn update
    
  4. Try to redeploy the same tag, enabling verbose output, with the following command :
    /root/quattor/scripts/build-tag.py --debug TAG-VALUE
    

If SVN cache is empty (not recognized as a valid SVN working copy by svn info), you need to identify last tag in tags branch of Quattor SCDB repository using svn ls.

Note: if you are still using the obsolete deployment script, build-tag.pl, most of the troubleshooting steps should apply but you need to adapt the configuration file and script names.

Troubleshooting SVN hook script

Most of the deployment problems, after initial configuration, are related to Deployment script. As the hook script mainly launches the deployment script, it is important to complete previous checks before troubleshooting SVN hook script.

The main cause of problems in the hook scripts is related to launching the deployment script from the hook script. The reason for the problems and the troubleshooting steps are different whether you use sudo or ssh to run the deployment script on the Quattor deployment server.

  • When using ssh, the main reason is invalid SSH keys for communication between SVN server and the Quattor server and invalid configuration. To troubleshoot this, log in to the Subversion server as the same user as your SVN server (generally Apache account) and try to ssh to the Quattor deployment server using the same account as the one used by the deployment script (typically root).
  • When using sudo, log in to the Subversion server as the same user as your SVN server (generally Apache account) and try to sudo under the deployment account (the one used to run the deployment script, typically root). One well-known issue with recent version of sudo is that you need to enable the use of sudo by processes without a tty (this must be done with visudo).

When you assessed the ability to launch the deployment script is working properly, try to execute the hook script under the SVN server account. This involves the following steps:

  1. Log on Subversion server as the same user as your Subversion server (generally Apache account).
  2. Retrieve the repository revision associated with the last deployment tag. For this, you need to run the following command (with tag_url the URL associated with the tag branch of the Quattor repository):
    svn log --limit 1 tag_url
    
  3. Go to the hooks directory of Subversion SCDB repository and run the following command (REPOSITORY_PATH is the filesystem path to the repository) :
    ./post-commit --debug REPOSITY_PATH REVISION
    

Archived Documentation

The information in this section is kept here for reference. It documents the configuration file format of the obsolete hook script and deployment script.

Hook script (obsolete)

This script requires a configuration file to be present, /etc/quattor-deploy.conf. The configuration file must contain only shell variable definition (sh syntax). The required variables are :

# Quattor server name
QUATTORSRV="quattorsrv.lal.in2p3.fr"

# Recipients of status mail and location for checkout.
NOTIFY="quattor-mgrs@lal.in2p3.fr"

Other supported variables are :

# Script on Quattor server to deploy new configuration
# Default : DEPLOYSCRIPT=/root/quattor/scripts/build-tag.pl

# User to run the deployment script
# Default : DEPLOYUSER=root

# Full paths to commands.
# Defaults are appropriate for RHEL/SL Linux.
# MAILER="/bin/mail"
# SVNLOOK="/usr/bin/svnlook"
# SSH="/usr/bin/ssh"

# Use sudo rather than ssh between SVN server and quattor deployment server
# Default: SSH
# USE_SUDO=1

# Default prefix for mail subject in case of error
# Default : ERROR_PREFIX='[Quattor-Deploy]'

Deployment Script (obsolete)

/etc/build-tag.conf allows to customize build-tag.pl according to local needs. It must be present. This file must contain key/value pairs (with '=' between key and value, keys are not case sensitive). Supported configuration options are :

  • debug : display additional information if non zero.
  • java_home : path of Java virtual machine.
  • java_version : alternative to java_home. Must match a directory under /usr/java. Default : $JAVA_HOME if defined.
  • svn_cache : location to use for SCDB repository local cache. Default : svncache at same level as the directory where the script is installed.
  • use_svn : if non zero, force the use of svn command even if jsvn exists. Default : use jsvn from svn_cache if it is present. Note : using jsvn may affect performances.
  • tags_svn_url (required) : SVN URL for SCDB repository tags branch

An /etc/build-tag.conf example is :

use_svn = 1
java_version = jdk1.6.0
tags_svn_url = http://grid280.lal.in2p3.fr/svn/quattor/tags
Last modified 14 years ago Last modified on Dec 2, 2009, 7:13:48 PM