WN Profile Cloning

TOC(inline)

Profile cloning is a technique to speed up profile compilation, taking advantage of nodes with an almost identical profile. The principle is that:

• A reference node is defined.
• On eligible nodes, the reference node compiled profile is included rather than recompiling the node profile. The only configuration parts not cloned from the reference node are hardware, network and file system configuration.

This results in a very significant speed up of the compilation: adding 100 clones typically add 10% to the compilation time of the reference node.

Configuration of Profile Cloning

Profile cloning is configured through a set of variables that must be defined very early in the configuration. The default template name to configure profile cloning is site/wn-cloning-config.tpl. Another template name can be used by defining variable PROFILE_CLONING_CONFIG_SITE. This template is typically located in site-specific templates or in cluster-specific templates.

Enabling Profile Cloning

Profile cloning is currently restricted to gLite WNs (since gLite 3.1). It is not enabled by default. Its use is controlled by 2 variables:

• PROFILE_CLONING_ENABLED: this variable must be true in the node profile for profile cloning to be used. It is typically configured as true by default for all nodes, in site/wn-cloning-config.tpl or PROFILE_CLONING_CONFIG_SITE.
• PROFILE_CLONING_DISABLED: this is a nlist with one entry per node for which you want to disable the use of profile cloning. The key must be the profile name of the node, without the profile prefix if defined. If the value is true, profile cloning will be disabled, even though PROFILE_CLONING_ENABLED is defined to true. Any other value will be ignored.

Defining the Reference Node

The reference node is defined with variable PROFILE_CLONING_REFERENCE_NODE. The value must match a profile template name (without the profile prefix if defined) in the same cluster as the nodes to be cloned. If you want to use the profile cloning in several clusters, you need to define one reference node per cluster.

Note: a current restriction is that only one reference node can be defined per cluster.

Defining the Eligible Nodes

Eligible nodes are defined by a regexp applied to the profile name (without the profile prefix if defined and without the extension). This regexp is defined with variable PROFILE_CLONING_ELIGIBLE_NODES. The regexp must use a Perl-compatible syntax.

Defining Profile Prefix

By default, the profile name is considered as matching the node name. If this is not the case, in particular if you use a prefix like profile_ for your profile names, you need to define it with the variable PROFILE_PREFIX.

Miscellaneous Configuration Options

Profile cloning is configured at a very early stage of the compilation process. As a result, it doesn't have access to the variables defined by the standard configuration process but needs to access directly some of specific site templates. If you don't use the standard names for these profiles, you should define the following variables (that are used by the standard configuration process and have the same defaults):

• SITE_DB_TEMPLATE: name of the template defining variables DB_MACHINES and DB_IP. Default: site/databases.
• SITE_GLOBAL_VARS_TEMPLATE: name of the template defining global variables for the site. Default: site/global_variables.
• SITE_FUNCTIONS_TEMPLATE: name of the template defining site specific functions. Default: site/functions;
• SITE_CONFIG_TEMPLATE: name of the template defining the base site configuration. Default: site/config.
• GLITE_SITE_PARAMS: name of the template defining gLite site parameters. Default: site/glite/config.
• FILESYSTEM_CONFIG_SITE: name of the template configuring the file systems. Default: filesystem/config.
• FILESYSTEM_LAYOUT_CONFIG_SITE: name of the template defining the file system layout, used by the standard file system configuration template. Default: site/filesystems/glite.

A typical definition of these variables for sites still using non-namespaced templates for these site-specific templates is:

variable SITE_DATABASES ?= 'pro_site_databases';
variable SITE_GLOBAL_VARIABLES ?= 'pro_site_global_variables';
variable SITE_FUNCTIONS ?= 'pro_site_functions';
variable SITE_CONFIG ?= 'pro_site_config';