Using Yum as an SPMA backend

TOC

Introduction

This document describes the internals of the integration of Yum and SPMA. User documentation will be provided in Github pages.

The main priority is to make the transition as smooth as possible for users. În the first stage, all changes are on the client side. This means no changes to any Pan schemas. In the second phase, some schemas may add fields for the user's benefit, but it should be backwards-compatible.

Whatever changes we do to SPMA schema or internals, we must port them into AII, which is going to be time consuming.

What ncm-spma does

ncm-spma, essentially, writes two files: /etc/spma.conf and /var/lib/spma-target.cf. Afterwards, it may need to transfer the control to the spma program, who actually deals with the packages.

What spma does

The spma program reads its configuration file and the list of desired packages, potentially manipulates some URLs, and passes control to SPM::RPMPPkgr.

Current problems

First of all, it's the obvious lack of dependency resolution. To avoid any dependency problems in Pan, we install way too much.

Also, there are bugs in some RPM bindings that limit the size of our transactions.

Proposal

We'll fork off ncm-spma completely. Updating the system will require running ncm-ncd --co spma, as is currently done. The new ncm-spma will:

1. Use /software/repositories to populate /etc/yum.repos.d.
   • In a future iteration, all fields allowed in a Yum repository will be allowed.
2. For packages with a fixed repository, include them in the includepkgs entry of the relevant repository.
3. For packages with no fixed repository, include them in the includepkgs entry of /etc/main.conf.
4. Compute the set difference between rpm -qa and /software/packages.
5. Execute yum shell to:
   1. Remove the difference (these are outdated packages), unless userpkgs is set.
   2. Install all entries in /software/packages

In future iterations, version and architecture of any given package will be optional.

The spma command will become an alias for ncm-ncd --co spma. The rpmt-py and spma packages will become deprecated.

Users willing to go back to the previous behaviour can just downgrade ncm-spma, and it will just work.

Future work: reverting to a previous state

The only drawback is reverting to a previous state. Imagine that you deployed tag a, which introduced dependency x-1.2. Then, tag n+1 upgrades x to 1.4. For watever reasons, you redeploy a, and all the configuration is replayed. But x is still in 1.4, when we expected 1.2!!.

To solve this, we'll make use of the Yum history feature. We'll store in a file the deployed tags, and associate them with a transaction ID. If ncm-spma finds that the tag had already been deployed, it will just undo all the subsequent transactions!

The map of tags to transactions can be stored in a JSON file, since it's expected to be small (a few hundreds of transaction at most). Something like this:

Error: Failed to load processor text/x-javascript

No macro or processor named 'text/x-javascript' found

Question: Which portion of the profile contains this tags? At UGent, we use /system/quattorid.

Note: This feature requires Yum 3.2.29, which is not shipped (but can be built) on SL5.