wiki:Obsolete/Development/Templates/CheckDeps

Quattor Tool Catalog

Introduction

One of the annoying things about quattor is that you can't check that a package set defined in a profile is consistent, i.e. that it doesn't require any more packages added to resolve dependencies. This can result in a frustrating sequence of "tweak package list, deploy profiles, run SPMA, check for dependency errors". The problem is that dependency information is not available when compiling profiles.

However, there is a tool that's very good at resolving dependency problems, and it's called YUM. The following recipe gives details of using YUM and a python utility called checkdeps (available in the subversion repository) to detect dependency problems and suggest a solution. I've found that it takes less than 15 seconds to check a profile, so you probably only want to do this for profiles that are causing problems, or as a final pre-deployment check.

Quick start

  • create YUM repositories in all your quattor repository directories
  • install yum and python-lxml on your development machine
  • get checkdeps
  • run checkdeps, pointing it at the directory where profiles are located:
     ./checkdeps -l build/xml
    

Server setup

Create YUM repositories

You will need to turn your existing quattor repositories into YUM repositories. Make sure that createrepo is installed on your repository server and then issue this command for every directory used as a target for a quattor software repository:

 createrepo <repository root>

For example, to convert the repository ca rooted at /var/www/html/ca issue this command:

 createrepo /var/www/html/ca

Client setup

YUM client

If YUM isn't on the machine where you normally develop Quattor profiles, you will need to install it. Note that that checkdeps has been tested with YUM version 3.2.19 and 3.2.21, although versions newer than 3.2.8 should work. If you don't have YUM or the version in your distribution is too old (below 3.2.8), you'll have to upgrade Yum.

You can create a directory for YUM to store its metadata in. Any directory that you can write to will do. This can be passed in to checkdeps as its working directory. If you don't set up a directory, then checkdeps will create one at ~/.quattor/yumroot

Get checkdeps and dependencies

You will need the python-lxml library installed. It may be available in your distribution, or there are RPMs at DAG. checkdeps is available in the SCDB distribution at checkdeps

Running checkdeps

If you simply want to check all profiles, you just need to pass checkdeps the location of the XML files as follows:

 checkdeps -l build/xml

To check a particular file in that location:

 checkdeps -l build/xml -x wn.example.org

Here is the output when your profile is consistent:

 $ checkdeps -x cagnode22.cs.tcd.ie.xml 
 checkdeps: 1 profile(s) to check in /home/childss/workspace/GI-SCDB/trunk/build/xml/csTCDie
 checkdeps: cagnode22.cs.tcd.ie.xml - profile seems OK.

If we remove the make and glib packages from the profile and run again, we get this:

 $ checkdeps -x cagnode22.cs.tcd.ie.xml -l `pwd`
 checkdeps: 1 profile(s) to check in /home/childss/svn/SCDB/trunk/utils/profiles
 checkdeps: cagnode22.cs.tcd.ie.xml rpm check error with transaction
 checkdeps: cagnode22.cs.tcd.ie.xml - YUM wants to add the following 2 package(s):

 Package                        Version              Arch     Needed by
 =================================================================================
 make                           3.81-1.1             x86_64   ypserv-2.19-3.x86_64
                                                              systemtap-0.5.14-1.el5.x86_64
                                                              efax-0.9-27.2.1.x86_64
                                                              nss_db-2.2-35.3.x86_64
                                                              openldap-servers-2.3.27-8.el5_1.1.x86_64
 glib                           1.2.10-20.el5        x86_64   gtk+-1.2.10-56.el5.x86_64
 
 checkdeps: cagnode22.cs.tcd.ie.xml - problem with profile

YUM detects a dependency problem and suggests a solution. The list of enabled repositories was generated from those contained in the profile, so the package suggested should be sensible.

If you add the -p flag, then checkdeps will output Pan lines adding the package (using pkg_repl) for easy cut-and-paste into the offending profile!

 checkdeps: Pan snippet to add missing packages:
 
 "/software/packages"=pkg_repl("make","3.81-1.1","x86_64");
 "/software/packages"=pkg_repl("glib","1.2.10-20.el5","x86_64");

Configuration

checkdeps takes a variety of configuration options, either via the command line or from a configuration file (~/.quattor/checkdeps.conf).

Option Default Command-line Config file
YUM config file location ~/.quattor/yumroot/yum.checkdeps.conf -c <file> [yum]config
SPMA target file - -t <file> -
XML profile - -x <file> -
YUM root dir ~/.quattor/yumroot -r <dir> [yum]root
Generate PAN True -p [config]output_pan
Debug level 0 -d <0..6> [config]debug_level
Location of profiles Current directory -l <path> [config]profile_dir
Pattern to select profiles *.xml -m <pattern> [config]match
External repository - -e <Tag:URL> -

A sample configuration file is as follows:

 $ cat ~/.quattor/checkdeps.conf 
 [config]
 generate_repos=True
 output_pan=False
 debug_level=0
 profile_dir=/home/childss/workspace/GI-SCDB/trunk/build/xml/csTCDie
 match=*.cs.tcd.ie.xml
 match=*.xml
 
 [yum]
 root=/home/childss/yumroot

Troubleshooting

No handlers could be found…

Yum changes the API from one version to another. This means, that you are using an old version and you should upgrade. This script has been tested on Yum versions for Fedora 8 and 9. You should use at least version 3.2.8, shipped with SL 5.2.

Errors on packages while checking dependencies

First, check your yum configuration. If it's all right, it means that you upgraded Yum following the steps above, but the repository metadata was created with an older version of createrepo. Update your createrepo to an stable version.

Version createrepo-0.4.11-3.el5 available for SL/EL5 is fine. Please note that yum 3.2.19 used in combination with createrepo 0.9.5 is known to cause problems.

Errors with repositories

If YUM repositories don't exist on the server, or can't be retrieved from the client, you will get errors like this:

 $ checkdeps  -l build/xml/
 checkdeps: 18 profile(s) to check in /home/childss/svn/SCDB/trunk/build/xml/
 checkdeps: bdii.example.org.xml error setting up YUM repository (check on server)
 checkdeps: bdii.example.org.xml problem retrieving YUM repository (check on server)
 checkdeps: bdii.example.org.xml - problem with profile

To troubleshoot this problem you can use the command:

 $ yum -c ~/.quattor/yumroot/yum.checkdeps.conf repolist

If you redefine YUM configuration file to something else than the default, use the appropriate path.

Nonsensical dependencies and suggestions

The local repository cache in yumroot may become corrupt - resulting in suggestions, for example, of packages from SL5 being recommended for SL4, or packages that are in the profile being reported as missing dependencies. A workaround is to delete

 $  ~/.quattor/yumroot/

and run checkdeps again.

TypeError: Can not create packages table: near "release": syntax error

If you get a Python traceback ending with:

 TypeError: Can not create packages table: near "release": syntax error

this is due to an incompatibility between certain versions of sqlite and yum, as explained in Red Hat Bug 481189. The affected versions are 3.6.8 to 3.6.10 and the issue is resolved in sqlite 3.6.11. This is fixed in Fedora 11 and Debian Sid. Ubuntu Jaunty users will need to install the sqlite3 package from Debian Sid.

Last modified 14 years ago Last modified on Mar 22, 2010, 5:06:57 PM