= Release Notes for SCDB =
[[TracNav]]

[[TOC(inline)]]

SCDB is a set of related tools to administer Quattor templates. This page describes new features introduced, known problems and displays last entries from Subversion log.


== Releases ==

The [source:SCDB/tags/pro production version] of SCDB Tools is can be downloaded from https://svn.lal.in2p3.fr/LCG/QWG/SCDB/tags/pro.

|| Date || Version || Description ||
|| 27/03/07 || [milestone:SCDB-2.0.0 2.0.0] || panc v7 as default compiler ||
|| 20/04/07 || [milestone:SCDB-2.1.0 2.1.0] || SVNKit upgrade (SVN 1.4 compatible) ||
|| 04/05/07 || [milestone:SCDB-2.1.1 2.1.1] || panc v7.2.0 ||
|| 21/05/07 || [milestone:SCDB-2.1.2 2.1.2] || New tag format, support for repository/ namespace ||
|| 19/06/07 || [milestone:SCDB-2.1.3 2.1.3] || panc 7.2.3, support for compilation of one cluster, bug fixes ||
|| 07/11/07 || [milestone:SCDB-2.1.4 2.1.4] || panc 7.2.4, fixed OS templates generation tools ||
|| 12/02/08 || [milestone:SCDB-2.1.5 2.1.5] || panc 7.2.9, fixed OS templates generation tools ||
|| 11/11/08 || [milestone:SCDB-2.2.0 2.2.0] || panc 8.2.3, checkdeps.py, sudo-based hook script ||
|| 12/07/09 || [milestone:SCDB-2.3.0 2.3.0] || panc 8.2.9, quatview, hiearchical arrangement of clusters ||
|| 23/08/09 || [milestone:SCDB-2.3.1 2.3.1] || Fixes for problems in [milestone:SCDB-2.3.0 2.3.0], rewrite of cluster group support, improvements to ant configuration ||
|| 13/9/09 || [milestone:SCDB-2.3.2 2.3.2] || Fixed `update.vo.config` (completely broken in previous 2.3.x and buggy in 2.2) ||

== Known Problems ==

=== 2.3.0/2.3.1: update.vo.config broken ===

Due to a mistake when SCDB 2.3.0 and 2.3.1 were packaged, the scdb-ant-utils task `VOConfigTask` implementing ant target `update.vo.config` is broken. There is no workaround, a fix will be released as soon as it is available.

'''This problem has been fixed in SCDB release [milestone:SCDB-2.3.2 2.2.3].''' The only changed file is `external/scdb-ant-utils/lib/quattor.jar`.

== New and Changed Features ==

=== 2.3.2: update.vo.config fixed ===

SCDB `ant` target `update.vo.config` was buggy in SCDB 2.2 series and [#a2.3.02.3.1:update.vo.configbroken completely broken] in SCDB 2.3 series. This has been fixed in SCDB [milestone:SCDB-2.3.2 2.3.2]. As a consequence of the fixes and some internal changes made to ensure that the account name associated with a given FQAN will never change, this may lead to some account name changes for some VOs. The creation of the new accounts will be properly handled by QWG templates but the NCM component used to manage accounts, `ncm-accounts`, is not able to removed the obsolete acccounts. It may be necessary (it is highly recommended on a CE) to remove these accounts, easily identifed as their name contains `fqan`. A template configuring a script to do it on every node where VO accounts are created is provided as an [source:templates/trunk/site/example/site/misc/purge_fqan_accounts example]. To use it, define variable `GLITE_BASE_CONFIG_SITE` to the name of this template.

=== 2.3.1: ability to generate a list of all configured package repositories added ===

It is now possible, when updating templates describing package repositories (`ant update.rep.templates`), to generate an additional template, `repository/allrepositories.tpl`, that defines the variable `ALL_REPOSITORIES`. This variable is a nlist where the key is the repository name and the value the repository information as expected by `/software/repositories`. See SCDB [/wiki/Doc/SCDB/Usage#ManagementofPackageRepositoryTemplates users guide] for details.


=== 2.3.1: ant default configuration can be customized with quattor.build.properties ===

It is now possible to use file `quattor.build.properties` in SCDB instance root directory (the same directory as the one where `quattor.build.xml` resides) to customize default configuration values used by SCDB Ant tools. For details, look at SCDB [/wiki/Doc/SCDB/Usage#ConfiguringSCDBAntTools users guide].

=== 2.3.1: fix for profiles-info.xml not properly updated ===

Due to changes introduced in SCDB [milestone:SCDB-2.3.0 2.3.0] for [#supportforhierarchicalarrangementofclusters initial support] of ''cluster groups'', `profiles-info.xml`, a file used by other Quattor components to know the list of existing node profiles on a deployment server, was updated too early and new profiles added where not relected in the list during the first deployment (#212). This problem has been fixed.

See [/wiki/Doc/SCDB/Usage#ClusterGroups cluster groups] documentation to know how to enable a specific deployment area per cluster group. Note that this feature is not fully integrated with standard deployment method, based on `ant deploy`.

=== 2.3.1: cluster groups support changes ===

[#supportforhierarchicalarrangementofclusters Initial support] of hierarchical arrangement of cluster, introduced in SCDB [milestone:SCDB-2.3.0 2.3.0] has been rewritten for a better integration and renamed ''cluster groups''. It is now fully documented in SCDB [/wiki/Doc/SCDB/Usage#ClusterGroups users guide].

=== 2.3.0: support for hierarchical arrangement of clusters ===

Using ant variable `cfg.siteprefix` (that must be defined in `quattor.build.properties`), it is possible to group clusters by ''sites'' in `clusters` directory. ''sites'' here doesn't refer to the usual `sites` directory but is a group name for clusters.

=== 2.3.0: panc upgraded to 8.2.9 ===

PAN compiler has been upgraded to version 8.2.9. In addition to fixes, this adds several new features:
 * Support for selective debug using ant variables `pan.debug.include` and `pan.debug.exclude`. Both variables can be defined with `-D` option in ant. Their value are a regexp matching a template namespace.
 * Ability to ignore some templates in dependency calculations. This is enabled by default for repository templates as normally a profile doesn't need to be rebuilt if there is not another change related to the repository template change (eg. a package replaced).

=== 2.3.0: src/utils renamed utils/ ===

All scripts and tools previously in `src/utils` have been moved to `utils`. A symlink `src/utils` has been created for backward compatibility.

=== 2.2.0: removal of obsolete tools ==)

Some obsolete tools have been removed from SCDB: lld2pkgs, xmldb2hw, xml2pkgs.

=== 2.2.0: sudo supported as replacement for ssh ===

In [milestone:SCDB-2.2.0 SCDB 2.2.0], it is possible to use `sudo` instead of `ssh` to launch the deployment script (`build-tag.pl`) from the SVN hook script. This is currently implemented as an alternative version of the SVN post-commit script, [source:templates/SCDB/trunk/src/hooks/post-commit.sudo post-commit.sudo].

=== 2.2.0: all external tools updated to last version ===

All SCDB external tools (in `external`) have been updated to last version available. This includes `svnkit`, `ant` and `Saxon`.

''Note: `svnkit` version provided is still 1.1.x, compatible with SVN client 1.4.x).''

=== 2.2.0: new ant target to generate VO parameters ===

[milestone:SCDB-2.2.0 SCDB 2.2.0] provides a new `ant` target, `update.vo.config`. This target allows to generate all the VO related templates from VO Identity Cards on [http://www.gridops.org CIC portal] for all registered VOs. 

=== 2.2.0: panc upgrade to v8, ===

[milestone:SCDB-2.2.0 SCDB 2.2.0] includes a major upgrade of panc compiler ([milestone:panc-8.2.3 8.2.3]). This version of panc is backward compatible with panc v7 but in v8, a few syntax constructs valid in v7 are now deprecated and must be fixed to ensure future compatibility with next major panc version, [milestone:panc-9.0.0 v9]. Check panc released notes for more details about these changes.

=== 2.1.5 : Script to generate OS errata added ===

A script to generate OS errata, [source:/SCDB/trunk/src/utils/misc/rpmErrata.tpl src/utils/misc/rpmErrata.tpl] has been added to SCDB  tools.

=== 2.1.5 : OS template generation tools fixed for RHEL and CentOS ===

OS template generation tools have been fixed to properly support non SL distribution. Support for Xen kernel has also been added. Options have been added to [source:/SCDB/trunk/src/utils/misc/buildOSTemplates src/utils/misc/buildOSTemplates] to support RHEL5-like distribuation layout.

=== 2.1.5 : build-tag.pl detects is another instance is running ===

In previous version, build-tag.pl didn't detect another instance was already running and this often resulted in
corruption of SVN cache leading to the need to check out SCDB again. This is fixed in version 2.1.5.

=== 2.1.5 : PAN compiler upgraded to 7.2.9 ===

PAN compiler has been upgraded to 7.2.9. This version mainly fixes a bug in self variable processing and is required by QWG Templates [milestone:gLite-3.1.0-3 gLite-3.1.0-3] and [milestone:gLite-3.0.2-14 gLite-3.0.2-14].

=== 2.1.4 : New script misc/buildOSTemplates ===

A new script to ease generation of templates for an OS distribution is available, [source:SCDB/trunk/src/utils/misc/buildOSTemplates src/utils/misc/buildOSTemplates]. Look at [wiki:Development/Templates/Generated#OSTemplates tool documentation] for more information.

=== 2.1.4 : New script misc/updateComponent ===

A new script to help updating NCM components from CVS is available, [source:SCDB/trunk/src/utils/misc/updateComponent source:SCDB/trunk/src/utils/misc/updateComponent src/utils/misc/updateComponent]. Look at [wiki:Development/Templates/Components#InstallingaNewVersion tool documentation] for more information.

=== 2.1.4 : New script misc/createPackagesTemplate ===

Feature provided by [source:SCDB/trunk/src/utils/CAs/create_template.pl src/utils/CAs/create_template.pl] have been extended. The new script is [source:SCDB/trunk/src/utils/misc/createPackagesTemplate misc/createPackagesTemplate]. Look at [wiki:Development/Templates/Generated#TrustedCAsTemplate CAs template] generation for more information.

[source:SCDB/trunk/src/utils/CAs/create_template.pl src/utils/CAs/create_template.pl] is now a wrapper to the new script.

=== 2.1.3 : Several message changes ===

To enhance usefulness of messages displayed by SCDB Ant Tools, several messages have been changed. It can be an issue if you have some local tools doing pattern matching on these messages. This affects only messages produced by SCDB Ant Tools, not by PAN compiler.

Main changes are :
 * Line indicating a modified file during `deploy` : now both local and remote changes are checked at the same time and the report includes if the changes are local and/or remote.
 * If `deploy` fails, the last message is `deploy aborted` instead of `tag aborted`
 * The cluster name being compiled used to be the cluster path and the useful part was at the end of the line. Now the message has the cluster name first and the path to the cluster template after.

=== 2.1.3 : Compiling only 1 cluster ===

With SCDB Tools 2.1.3, it is possible to compile only one cluster by adding the following option to `ant` command :
{{{
-Dcluster.name=name_of_cluster
}}}

`name_of_cluster` is the name of the directory under `cfg/clusters`.

''Note : this option is forbidden during `deploy`.''


=== 2.1.2 : Support for repository/ namespace ===

With SCDB Tools 2.1.2, the default with `update.rep.templates` is to generate a template name that uses agreed repository namespace '`repository/`). The namespace is prefixed to the template name as specified in the `name=` parameter in header comments. This is a change compared to behaviour in previous versions, where prefix `repository_` was added in front of the template name.

To ease the transition to namespace, new version of SCDB tools preserve the name of an existing template, as defined in `structure template` line. Thus to produce a non namespaced repository template, just add the `structure template` line with the appropriate name or modify the line created by `update.rep.templates`. This will be preserved during future changes.

=== 2.1.2 : New tag format requires a hook script upgrade ===

If you want to use the new hierarchichal tag format (e.g. yyyy/MM/dd-HH.mm.ss) introduced with SCDB 2.1.2, you need to update Subversion repository hook script and Quattor server script. Refer to [wiki:Download/SCDB#Installationofhookscriptandserverscript SCDB installation] instructions to know how to update them. In particular, take care to merge your site customizations into the hook script, as this script doesn't yet support an external configuration file.
 

=== 2.1.0 : Subversion 1.4 requires a new httpd version (SL3) ===

If you choose to upgrade your Subversion client to 1.4 or later to be consistent with new SVNKit version, on SL3.x machines, you need to install a new version of httpd 2.0.46. You can found the new RPM at URL http://quattorsrv.lal.in2p3.fr/packages/os/sl305-i386/updates/httpd-2.0.46-61.1.ent.i386.rpm.

On SL4.x machines, the standard httpd version works properly with new Subversion client.

=== 2.1.0 : SVNKit upgrade ===

SVNKit (former JavaSVN) has been upgraded to 1.1.2. This release is compatible with Subversion 1.4 client.

Before upgrading, you need to be aware that SVN 1.4 introduces a change in working area metadata format. As a consequence you cannot access/manage a working area with both a SVN 1.4 client and a previous version. There is no restriction in client/server versions combinations.

This may have a consequence on SCDB, depending on the client you use to do the SVN operations. This is because `ant deploy` task uses SVNKit and thus require a 1.4 working area metadata format. To solve any potention problems you need to upgrade the SVN client on the machine you use to administer SCDB (where you run `ant`) to version 1.4 or later.

As an alternative, you can use `jsvn` command provided with SVNKit instead of `svn`. The syntax is the same, refert to `svn help` for the command documentation. To use `jsvn` provided with SCDB tools, enter the following command :
{{{
external/svnkit/jsvn
}}}

=== 2.0.0 : default compiler changed to panc v7 ===

panc v7 (Java version of panc) is now the default compiler. Old compiler can still be used if necessary appending `.c` to ant task names, except for `deploy`.

Be sure to read instructions about [wiki:Doc/compiler/running running PAN compiler] if you experiment any strange behaviour or slowness with the new compiler.

=== 2.0.0 : configuration file for build-tag.pl ===

[source:SCDB/trunk/build-tag.pl build-tag.pl] is the script used to deploy configuration. It runs on the Quattor server. Version provided with 2.0.0 supports local customization through file [wiki:Doc/SCDB/Server#DeploymentScript /etc/build-tag.conf].

Use of new version supposes that you manually replace your existing script on the server with the new one provided in this release

== Known Problems ==

In addition to known problems documented here, look at [wiki:FAQ/SCDB SCDB FAQ].

=== 2.1.2 : update.rep.templates possible issue with empty repositories ===

SCDB tools v2 including 2.1.2 has a possible issue with empty repositories (#63). This happens if you removed from the associated template everything except comment header '''and''' `structure template` line. In this case, the template is not updated and the propeprty `name` is missing. This leads to an error in SPMA functions.

The easiest workaround is to also suppress the `structure template` line and update template name if the template is not namespaced.

This should be fixed in 2.1.3.

=== 2.0.0 : issue with accentuated characters ===

The new PAN compiler (v7) encodes non ASCII characters in UTF-8. Unfortunatly, NCM components (particularly ncm-filecopy, and CCM are not prepared to [wiki:ReleaseNotes/panc#UTF-8encodingofnonASCIIcharacters handle properly UTF-8] encoded characters. The workaround is to remove these non ASCII characters from the templates (they are generally comments).

This problem also affects CCM before release 1.5.13. If after upgrade to SCDB 2.0.0, CCM complains about illegal characters, you need either to revert to previous version of SCDB and upgrade QWG templates first to 3.0.2-11 or later (see QWG Templates [wiki:ReleaseNotes/gLite-3.0#gLite-3.0.2-11:upgradetopancv7recommended Release Notes]), or to temporaly comment out all non ASCII characters in your templates (generally they are part of comments).

This will be fixed in a future release (#17). 

== Change Log ==

[[ChangeLog(SCDB/trunk,20)]]