| | 1 | [[TracNav]] |
| | 2 | |
| | 3 | == Introduction == |
| | 4 | |
| | 5 | One of the annoying things about quattor is that you can't check that a package set defined in a profile is consistent, i.e. that it doesn't require any more packages added to resolve dependencies. This can result in a frustrating sequence of "tweak package list, deploy profiles, run SPMA, check for dependency errors". The problem is that dependency information is not available when compiling profiles. |
| | 6 | |
| | 7 | However, there is a tool that's very good at resolving dependency problems, and it's called YUM. The following recipe gives details of using YUM and a python utility called '''checkdeps''' (available in the [http://quattor.svn.sourceforge.net/viewvc/quattor/trunk/util/checkdeps subversion repository]) to detect dependency problems and suggest a solution. I've found that it takes less than 15 seconds to check a profile, so you probably only want to do this for profiles that are causing problems, or as a final pre-deployment check. |
| | 8 | |
| | 9 | === Quick start === |
| | 10 | * create YUM repositories in all your quattor repository directories |
| | 11 | * install yum and python-lxml on your development machine |
| | 12 | * get [https://quattor.svn.sourceforge.net/svnroot/quattor/trunk/util/checkdeps/checkdeps checkdeps] |
| | 13 | * run checkdeps, pointing it at the directory where profiles are located: |
| | 14 | {{{ |
| | 15 | ./checkdeps -l build/xml |
| | 16 | }}} |
| | 17 | |
| | 18 | == Server setup == |
| | 19 | |
| | 20 | === Create YUM repositories === |
| | 21 | |
| | 22 | You will need to turn your existing quattor repositories into YUM repositories. Make sure that '''createrepo''' is installed on your repository server and then issue this command for every directory used as a target for a quattor software repository: |
| | 23 | {{{ |
| | 24 | createrepo <repository root> |
| | 25 | }}} |
| | 26 | |
| | 27 | For example, to convert the repository `ca` rooted at `/var/www/html/ca` issue this command: |
| | 28 | {{{ |
| | 29 | createrepo /var/www/html/ca |
| | 30 | }}} |
| | 31 | |
| | 32 | == Client setup == |
| | 33 | |
| | 34 | === YUM client === |
| | 35 | |
| | 36 | If YUM isn't on the machine where |
| | 37 | you normally develop Quattor profiles, you will need to install it. Note that that checkdeps has |
| | 38 | been tested with YUM version 3.2.19 and 3.2.21, although versions newer than 3.2.8 should work. If you don't have YUM or the version in your distribution is too old (below 3.2.8), you'll have to [http://ftp.scientificlinux.org/linux/scientific/5x upgrade Yum]. |
| | 39 | |
| | 40 | You can create a directory for YUM to store its metadata in. Any directory that you can write to will do. This can be passed in to checkdeps as its working directory. If you don't set up a directory, then '''checkdeps''' will create one at `~/.quattor/yumroot` |
| | 41 | |
| | 42 | === Get checkdeps and dependencies=== |
| | 43 | |
| | 44 | You will need the python-lxml library installed. It may be available in your distribution, or there are RPMs at [http://dag.wieers.com/rpm/packages/python-lxml/ DAG]. checkdeps is available in the SCDB distribution at [https://quattor.svn.sourceforge.net/svnroot/quattor/trunk/util/checkdeps/checkdeps checkdeps] |
| | 45 | |
| | 46 | == Running checkdeps == |
| | 47 | If you simply want to check all profiles, you just need to pass '''checkdeps''' the location of the XML files as follows: |
| | 48 | {{{ |
| | 49 | checkdeps -l build/xml |
| | 50 | }}} |
| | 51 | |
| | 52 | To check a particular file in that location: |
| | 53 | {{{ |
| | 54 | checkdeps -l build/xml -x wn.example.org |
| | 55 | }}} |
| | 56 | |
| | 57 | Here is the output when your profile is consistent: |
| | 58 | {{{ |
| | 59 | $ checkdeps -x cagnode22.cs.tcd.ie.xml |
| | 60 | checkdeps: 1 profile(s) to check in /home/childss/workspace/GI-SCDB/trunk/build/xml/csTCDie |
| | 61 | checkdeps: cagnode22.cs.tcd.ie.xml - profile seems OK. |
| | 62 | }}} |
| | 63 | |
| | 64 | If we remove the make and glib packages from the profile and run again, we get this: |
| | 65 | {{{ |
| | 66 | $ checkdeps -x cagnode22.cs.tcd.ie.xml -l `pwd` |
| | 67 | checkdeps: 1 profile(s) to check in /home/childss/svn/SCDB/trunk/utils/profiles |
| | 68 | checkdeps: cagnode22.cs.tcd.ie.xml rpm check error with transaction |
| | 69 | checkdeps: cagnode22.cs.tcd.ie.xml - YUM wants to add the following 2 package(s): |
| | 70 | |
| | 71 | Package Version Arch Needed by |
| | 72 | ================================================================================= |
| | 73 | make 3.81-1.1 x86_64 ypserv-2.19-3.x86_64 |
| | 74 | systemtap-0.5.14-1.el5.x86_64 |
| | 75 | efax-0.9-27.2.1.x86_64 |
| | 76 | nss_db-2.2-35.3.x86_64 |
| | 77 | openldap-servers-2.3.27-8.el5_1.1.x86_64 |
| | 78 | glib 1.2.10-20.el5 x86_64 gtk+-1.2.10-56.el5.x86_64 |
| | 79 | |
| | 80 | checkdeps: cagnode22.cs.tcd.ie.xml - problem with profile |
| | 81 | }}} |
| | 82 | |
| | 83 | YUM detects a dependency problem and suggests a solution. The list of |
| | 84 | enabled repositories was generated from those contained in the profile, |
| | 85 | so the package suggested should be sensible. |
| | 86 | |
| | 87 | If you add the -p flag, then checkdeps will output Pan lines adding |
| | 88 | the package (using `pkg_repl`) for easy cut-and-paste into the |
| | 89 | offending profile! |
| | 90 | {{{ |
| | 91 | checkdeps: Pan snippet to add missing packages: |
| | 92 | |
| | 93 | "/software/packages"=pkg_repl("make","3.81-1.1","x86_64"); |
| | 94 | "/software/packages"=pkg_repl("glib","1.2.10-20.el5","x86_64"); |
| | 95 | }}} |
| | 96 | |
| | 97 | == Configuration == |
| | 98 | checkdeps takes a variety of configuration options, either via the command line or from a configuration file (`~/.quattor/checkdeps.conf`). |
| | 99 | |
| | 100 | || Option || Default || Command-line || Config file |
| | 101 | || YUM config file location || `~/.quattor/yumroot/yum.checkdeps.conf` || -c <file> || [yum]config || |
| | 102 | || SPMA target file || - || -t <file> || - || |
| | 103 | || XML profile || - || -x <file> || - || |
| | 104 | || YUM root dir || `~/.quattor/yumroot` || -r <dir> || [yum]root || |
| | 105 | || Generate PAN || True || -p || [config]output_pan || |
| | 106 | || Debug level || 0 || -d <0..6> || [config]debug_level || |
| | 107 | || Location of profiles || Current directory || -l <path> || [config]profile_dir || |
| | 108 | || Pattern to select profiles || *.xml || -m <pattern> || [config]match || |
| | 109 | |
| | 110 | A sample configuration file is as follows: |
| | 111 | {{{ |
| | 112 | $ cat ~/.quattor/checkdeps.conf |
| | 113 | [config] |
| | 114 | generate_repos=True |
| | 115 | output_pan=False |
| | 116 | debug_level=0 |
| | 117 | profile_dir=/home/childss/workspace/GI-SCDB/trunk/build/xml/csTCDie |
| | 118 | match=*.cs.tcd.ie.xml |
| | 119 | match=*.xml |
| | 120 | |
| | 121 | [yum] |
| | 122 | root=/home/childss/yumroot |
| | 123 | }}} |
| | 124 | |
| | 125 | == Troubleshooting == |
| | 126 | |
| | 127 | === No handlers could be found... === |
| | 128 | Yum changes the API from one version to another. This means, that you |
| | 129 | are using an old version and you should upgrade. This script has been |
| | 130 | tested on Yum versions for Fedora 8 and 9. You should use at least |
| | 131 | version 3.2.8, shipped with SL 5.2. |
| | 132 | |
| | 133 | === Errors on packages while checking dependencies === |
| | 134 | First, check your yum configuration. If it's all right, it means that |
| | 135 | you upgraded Yum following the steps above, but the repository metadata |
| | 136 | was created with an older version of createrepo. Update your |
| | 137 | createrepo to an stable version. |
| | 138 | |
| | 139 | Version createrepo-0.4.11-3.el5 available for SL/EL5 is fine. Please note that yum 3.2.19 used in combination with createrepo 0.9.5 is known to cause problems. |
| | 140 | |
| | 141 | === Errors with repositories === |
| | 142 | If YUM repositories don't exist on the server, or can't be retrieved from the client, you will get errors like this: |
| | 143 | {{{ |
| | 144 | $ checkdeps -l build/xml/ |
| | 145 | checkdeps: 18 profile(s) to check in /home/childss/svn/SCDB/trunk/build/xml/ |
| | 146 | checkdeps: bdii.example.org.xml error setting up YUM repository (check on server) |
| | 147 | checkdeps: bdii.example.org.xml problem retrieving YUM repository (check on server) |
| | 148 | checkdeps: bdii.example.org.xml - problem with profile |
| | 149 | }}} |
| | 150 | |
| | 151 | To troubleshoot this problem you can use the command: |
| | 152 | {{{ |
| | 153 | $ yum -c ~/.quattor/yumroot/yum.checkdeps.conf repolist |
| | 154 | }}} |
| | 155 | |
| | 156 | If you redefine YUM configuration file to something else than the default, use the appropriate path. |
| | 157 | |
| | 158 | === Nonsensical dependencies and suggestions === |
| | 159 | |
| | 160 | The local repository cache in yumroot may become corrupt - resulting in suggestions, for example, of packages from SL5 being recommended for SL4, or packages that are in the profile being reported as missing dependencies. A workaround is to delete |
| | 161 | {{{ |
| | 162 | $ ~/.quattor/yumroot/ |
| | 163 | }}} |
| | 164 | |
| | 165 | and run checkdeps again. |
| | 166 | |
| | 167 | === TypeError: Can not create packages table: near "release": syntax error === |
| | 168 | If you get a Python traceback ending with: |
| | 169 | {{{ |
| | 170 | TypeError: Can not create packages table: near "release": syntax error |
| | 171 | }}} |
| | 172 | this is due to an incompatibility between certain versions of sqlite and yum, as explained in [https://bugzilla.redhat.com/show_bug.cgi?id=481189 Red Hat Bug 481189]. The affected versions are 3.6.8 to 3.6.10 and the issue is resolved in sqlite 3.6.11. This is fixed in Fedora 11 and Debian Sid. Ubuntu Jaunty users will need to install the sqlite3 package from Debian Sid. |
