= Creating gLite Templates = [[TracNav]] [[TOC(inline)]] This page contains guidelines and receipees to create and maintain templates related to gLite services. This is not The Definitive Guide as there is no general rule that works with every service. Almost every gLite service has some specifics. == gLite Service Templates == QWG templates for gLite are organized in two categories of services : * ''high-level'' services : the services exposed to the user like CE, SE, BDII, WMS... Configuration for these services are in `glite/` directory of grid templates for a specific gLite version. * ''low-level'' services : all the underlying services used by ''high-level'' services, like gridmap, globus, gip or configuration shared between several services like security, torque, nfs... For complex services, like WMS, it may happend there is both a high-level service and a low-level service called `wms`. In this case, the low-level service is service configuration part that is used by other high-level service. The bottom line is that high-level services are not cross-referencing each others, with the exception of BDII high-level service which is included in most of the others. In addition to templates describing services, there is generally one template called a ''machine type'' template, sitting in `machine-types` directories of grid templates. This template does everything required to configure a machine with the corresponding high-level services, including OS configuration. When a high-level service has several configuraion variants (e.g. WMS and/or LB, DPM head node or disk servers), there is generally only one machine-type template with a variable allowing to select the appropriate variant for one specific machine. Each service description is arranged basically around the same layout, with generally at least 3 templates for high-level services: * `config.tpl`: this templates does all the configuration specific to the service, whether high-level or low-level but doesn't take care of loading packages used by the service and is not responsible for configuring other services used by the service. In some services, the name of the template may be different, if more appropriate. * `service.tpl`: this template configures every low-level service used by the high-level service, including the specific configuration (`config.tpl`) and RPMs (`rpms/config.tpl) for the service. This template exists only for high-level services and is the main ''entry point'' for configuring the service. * `rpms/config.tpl` : this template is responsible for adding all the packages (RPMs) used by the service. It seldom exists in low-level services but is always present in high-level service. Generally this template mainly includes another template which does the real work and is architecture specific. The architecture specific templates must be in a directory `rpms/arch` (e.g. `rpms/x86_64`) and is generally called `config.tpl` but may have another name if more appropriate, in particular in services supporting several variants (like DPM, dCache...). == Where to Find Documentation == This is one of the difficulties... There is no central place where you can expect to find the documentation about configuring a service. But there are a few places where you can expect to find some useful information or pointers, mainly: * [http://glite.web.cern.ch/glite/documentation/default.asp gLite web site]: there is generally at least some basic information on how to configure the service, most of the documentation is more about using or administering the service. But it may be worth to have a look... * CERN Twiki pages: there you may find some useful information for services heavily used by LCG, like WMS, FTS, DPM... Finding the URLs is not always trivial, Google is your best friend there... The most useful source of information to start is probably YAIM scripts. This is in fact just one example on how to configure a service, there are sometimes other variants and for several services QWG doesn't do exactly the same way as YAIM to provide more flexibility. But at least, this is a configuration generally working that can be used as a starting point. YAIM scripts are just shell scripts and are in 2 directories: * `/opt/glite/yaim/node.d`: there is one script for each machine type. This is pretty much an equivalent for QWG templates in `service.tpl`. It gives the full list of YAIM scripts used to configure all the services required for this machine type and is a good source to identify what must be in `service.tpl`. * `/opt/glite/yaim/scripts`: there is one script for configuring each individual service. This is where to look to get an idea of what should be done in `config.tpl` for a high-level or low-level service. Last but not least, another useful source of information is [source:templates/tunk/grid/glite-3.1/glite templates] used to configured another gLite services. It may help to understand the layout and identify what is generally done in one category of templates. == How to Start with a New gLite Service == One important design choice in QWG templates is to try as much as possible to avoid interdependencies between services to ensure the final configuration is not dependent of the order the services have been configured. This is not always possible (e.g. BDII and GIP are by design quite inter-dependent) and in this case it must be clearly documented in the templates. ''Note: QWG configuration relies heavily on global variables to customize a service configuration. It is important to ensure there is no conflict between variables used by two different services. The good practice is to prefix any variable used by a service by something identifying unambiguously the service like `WMS_`, `LB_`, `SEDPM`, `BDII`...'' === RPMs === Each gLite service is released with an official list of RPM, available on [http://glite.org gLite site]. The easiest way to get to this list is general by the page about releases where there is a link to high-level services available. From there you should get easily to the section with RPM list. The RPM list provided is generally reasonably accurate and complete. This RPM list must be downloaded in text format (there is a URL for this in every gLite RPM list) and then processed by [source:SCDB/trunk/src/utils/misc/createPackagesTemplate src/utils/misc/createPackagesTemplate] to produce the corresponding template in appropriate `rpms/arch` directory. Be sure to download the the RPM list for the appropriate architecture, when several are supported, as the version numbers are not always the same for each architecture. Use `src/utils/misc/createPackagesTemplate --help` for detailed information about options supported by this script. Main options are: script provided with SCDB. The steps are : * `--rpmlist file`: name of the file containing the download RPM list. * `--template file`: name of the template to create. * `--namespace ns`: PAN namespace to use in the template An example to create RPM list for the WN i386: {{{ src/utils/misc/createPackagesTemplate --rpmlist glite-WN-3.1.0-1.html \ --template cfg/grid-3.1/glite/wn/rpms/i386/rpms.tpl \ --namespace glite/wn/rpms/i386 }}} Once created, the generated templates must be tweaked to add missing dependencies if any but mainly to remove any RPM listed which is provided by the OS distribution. This is very important for the RPM list to be independent of a particular OS version. The RPMs provided by the OS must be moved to another template in templates specific to an OS version, in the `config/glite/3.1/` generally referred as ''OS hooks''. Most of the OS-provided RPMs are already present in `base.tpl` included in any gLite machine type. If some are missing, an OS hook template specific to the service must be created. Another common modification to the generated RPM list is to change architecture for RPMs not `noarch` to `PKG_GLITE_DEFAULT`. This helps copying RPM list from one architecture to another when this is useful. After the RPM list for each supported architecture is ready, it is necessary to create `rpms/config.tpl`. The easiest is to copy an existing one in another service as this is generally pretty much the same in every service. This template is the right place to force architecture used for the service independently of the machine OS. This is generally done by defining variable `PKG_GLITE_DEFAULT` to the appropriate architecture (must match the directory name under `rpms`, currently `i386` for 32-bit architecture and `x86_64` for 64-bit architecture). === service.tpl === `service.tpl` is responsible for configuring all related services and including the service specific configuration and its required packages. Packages (`rpms/config.tpl`) are generally included at the beginning of `service.tpl`. Service specific configuration (`config.tpl) is generally done last. But there is no strict rule, it may depends on some specific service requirements. For the services which are publishing their resources, `service.tpl` is in charge of configuring the BDII for the service. This is done with a very standard and generic pan code that must be copied from an existing service. In addition, to these steps, `service.tpl` is often in charge of executing template corresponding to `xxx_CONFIG_SITE` variable where `xxx` is a service identifier ( `WMS`, `UI`, `CE`...). This template can be provided by a site to do some site specific configuration for the service. Depending on services, there is no strict rule, this site specific configuration may be done before or after the standard service configuration. It mainly depends on whether this template is used to define default values for variables (in this case it must be before) or to add some specific actions (in this case it is often better done after). The main advice to build `service.tpl` is to start with `service.tpl` from an existing service with quite similar requirements in term of other service dependencies and refine it, based on documentation and YAIM script in `/opt/glite/yaim/node.d`. === config.tpl === components === Machine type === VO config == Documenting Configuration of a gLite Service == Whether configuration for a gLite service is modified/enhanced or wether configuration for a new service is added to QWG, it is '''very important''' to document them '''immediatly''' on the [lcgqwg:wiki:WikiStart LCG QWG wiki]. There are potentially 2 places where the change/addition must be documented: * [wiki:Doc/gLite/TemplateCustomization gLite Customization]: this page is an How-To page describing configuration of a particular service. Main entries in this page are mainly high-level services, with a few exceptions like NFS, Torque/MAUI, MPI. The most important information there is the officially supported variables available to customize the service. Information is added here as soon as (sometimes a little bit before!) the new configuration is committed to trunk. * [wiki:ReleaseNotes/gLite-3.1 Release Notes]: this page is a log (in reverse chronological order) of every major change/addition to the templates. It should be a short description linking to [wiki:Doc/gLite/TemplateCustomization gLite Customization] for details. Every entry in the log starts with the version of QWG templates where the change was introduced. The version used when the entry is added, as explained in the page, is always the next version to be released : don't wait for the official release to add entries!