= Quattor Workshop - CNAF (Bologna) - 30/9/2013-2/10/2013 = [[TracNav]] [[TOC(inline)]] [http://indico.cern.ch/conferenceTimeTable.py?confId=252955#all.detailed Agenda] == Project Status - L. Munoz == Survey ran before the workshop * Significant improvement compared to previous ones * QWG : many were not able to move to QWG or forked a long time ago. We need to think more heavily at it. * Aquilon adoption making progress (2-3 sites) * Community support is perceived as good * Documentation: still the weak point, were not able to make significant progresses * LAL wiki: certificate problem still perceived as a barrier, a developer only resource == Core modules == === Configuration Modules - L. Munoz === Releases * Count : nine releases, 3 of them tagged as stable * New releases are not mandatory * Need a releaser manager backup for Luis Config modules: main activity is on the core components * Not much on grid components or on daemons * ncm-openldap and ncm-cups were resurrected New features * Preliminary support for SSSD in authconfig * YUM-based SPMA ready for production * Hooks for ncm-ncd before and after execution of components Metaconfig stuff * Ugent has schemas and templates for many services: would like to share them * But we can hardly promise backward compatibility Work in progress * Policy-based component dispatching in ncm-ncd * Solaris port * ncm-network: still not much progress === ncm-ncd hooks and policy-based component scheduling === See [/wiki/Development/PrePostActionsNCMNCD proposal]. Goal: execute some arbitrary actions before running all components and after. * These actions will have no access to node profile * Specified by `run-init` and `run-finalize`in `/etc/ncm-ncd.conf` * An error during `run-init` will prevent executing components but `run-finalize` will run Hooks may be used to implement site-specific policy for component scheduling, allowing to filter out some components from the initial list based on some local criteria * But what to do with dependencies if one is filtered out? Can be very complicated * Probably better/enough to implement a "everything or nothing" strategy: at least use case can be cleanly defined Some use cases for executing only part of the configuration changes for a component. Proposal: * Split component configuration into 2 different parts/subtree, e.g. `/software/components/filecopy` and `/software/components/filecopy-magic` * Bind `/software/components/filecopy-magic` to `type_filecopy` * Define a component alias name in `filecopy-magic` saying it will use component `filecopy` to execute the action * New field in `deps` structure * Filter out `filecopy-magic` in run-init hook * See https://github.com/quattor/configuration-modules-core/issues/112 for discussion/implementation follow-up === CDBSQL - Z. Sebestyen === Used at MS to produce/update a SQLite db every 2 hours with the AQDB contents: used by users to query configuration rather than overloading the broker. * Reused an older API from MS to query the DB Code will be pushed to SF at some point * Still some concerns with pushing directly to GitHub Still need to review if it could be integrated with QuatView to replace the internal SQL generator in QuatView and deliver a generic web front-end. Also, may look at using a NoSQL db rather than SQLite. * May give some perf advantage for querying the db but may a price for the update == Maker Quattor Easier == === Introduction - D. Zilaskos === Painful path for a new Quattor user * Go to website and click download: not exactly what was being looked for... * Try documentation: suggest to go to overview... just marketing blurb... Should not dream of producing a polished documentation as we failed for so long... but we can make things much easier if not very easy. Goal should be to install a workable Quattor installation for testing in 30 mn! * Install from Quattor YUM repo * `yum install quattor-server` * `yum install quattor-client`? * Sample templates usable with minimum editing (host name, mac address) to install/bootstrap a first machine * For the first step, could be take control of already existing machines without doing the initial installation to avoid AII/kickstart difficulties * Panc book: improve the tutorial part Documentation may be organized in 2 different parts: * Getting started: build/describe an optimized procedure for a user to start with Quattor both for evaluating AND setting up a production infrastructure (could then be used to produce an appliance) * Prerequisite hardware/site config * Install server: prefer Aquilon if mature enough. We don't want to attract new sites to SCDB. * Take control of a client (already installed): should include a basic configuration description that is as less intrusive as possible for the existing configuration * Mini-demonstration illustrating how to change some service configuration, for example ssh keys, account creations, add a RPM * Next steps * How to find docs and examples * Initial installation: AII/kickstart customization * Security * Customizing node configurations: common services and recipes * Document how to execute an arbitrary command with existing config modules like filecopy or metaconfig Put documentation on GitHub: start with a non public (review) section * Review how to improve the documentation structure on gitHub: see documentation discussion later during the workshop Documentation: activate robot indexing for Quattor wiki hosted at LAL Timeline * Getting Started done at next workshop (April 2014) * Next steps after Encourage adoption of new YUM-based ncm-spma: make it clear that YUM is used in a completely standard way and that people may/must refer to YUM docs for details on advance YUM management (like mirroring). === QWG Status - G. Philippon === Recent changes * No more release for more than 1 year * No update to branches: all changes in the trunk * standard/ almost stable: most things are in fact imported from other places Current trunk status * Partly based on Quattor 13.1: most components not yet updated (validation required) * Support SL5 and SL6 3 different subsets shipped together * "Standard" templates * Provide a core node type without any grid dependency * Should be really standard: can it be the case for a Web server? * Grid templates * Templates generated from other Quattor components like AII or config modules StratusLab templates delivered separately (git repo) * Support both ncm-spma v2 and v3 Work in progress * Remove templates related other Quattor components from standard/ * Create a new namespace quattor/ with one sub-namespace per Quattor version: easier to move from one version to another one * YUM-based ncm-spma * Move to GitHub: do we keep the history? To be done but not yet handled by lack of time * Namespace structure of OS templates * Integration of Aquilon * NFS configuration independent of grid templates GitHub migration based on several repositories, no history kept * templ-library-standard: standard/ without the stuff imported from other places and without monitoring * monitoring not migrated * templ-library-core: templates imported from other Quattor components (config modules, AII, schema, Pan types...). Generated by the release process ideally, else some SCDB scripts may help (e.g. updateComponents) * master branch + 13.1 branch (ncm-spma v2) * Add standard/pan and standard/quattor other than AII * templ-library-grid: grid templates * 1 branch per major grid MW version * templ-library-stratuslab: current strastuslab repository * templ-library-examples: current clusters/ and sites in SVN * Minor issue: how to keep a "unified view" of these separate repositories for pushing back from a unique place like in the past * git submodule are not appropriate for this * git subtree: requires additional options when pushing but may be explored * Luis will try to find some examples === Core Quattor Schema === standard/pan and standard/quattor general templates were not migrated from SF SVN: need to be added to template-library-core Some suff added/changed for Aquilon: need to merge asap * Mostly additions but a few mods (network part in particular) == Actions == Documentation * Getting Started section on quattor.org * Allow indexing of Trac wiki by robots * Move to quattor.org parts of general interest on QWG wiki Configuration modules * Implement ncm-ncd hooks * Add support for component aliasing