Conary:Dynamic Tags

From rPath Wiki

See the Dynamic Tags section of Repository-Based System Management Using Conary for an introduction to the purpose of tags.

With the exception of certain file types (such as shared libraries), files that have tags attached to them will be processed by tag handlers. A tag name should describe the file, not the action to take based on the file; for example, <file> is-a <tag>

Tagging Files in Recipes

For most tags, recipes that create files that would be tagged already have the trove that provides the tag description file in their buildRequires list. For those tags, and for the special tags handled inside Conary (see below), you do not have to touch recipes.

However, if your recipe does not have a build requirement for the trove that would cause your file to be tagged, you should add a TagSpec line. One common example is initscripts. Very few packages that provide initscripts have a real build dependency on chkconfig, the package that provides the initscript tagdescription. Rather than adding 'chkconfig:runtime' to buildRequires, you should explicitly tag your initscripts with:

    r.TagSpec('initscript', '%(initdir)s/')

Needing to provide the TagSpec line violates the general rule of avoiding boilerplate in recipes, but it helps avoid two other evils:

• Putting distribution-defined bits in Conary proper
• Build dependency loops

Prerequisites

Prerequisites for package installation are merely runtime dependencies for the tag handler. So use lines like:

    r.Requires('%(bindir)s/foo', '%(taghandlerdir)s/')
    r.Requires('bar:runtime', '%(taghandlerdir)s/thistaghandler')

The first one says that every taghandler that the recipe installs should depend on the file /usr/bin/foo (be careful -- you can only require files that are provided. In modern versions of conary, all files in bin/ directories are provided, but it's good to double check that some package has that file listed in its provides section.) The second form is specific to one taghandler and has a trove requirement for the 'bar:runtime' component. Most recipes provide only one taghandler, so you can use the simple filterexp of '%(taghandlerdir)s/' as a shorthand.

Tag Catalog

Normal Tags

These are the tags defined in the @rpl namespace:

Special Tags

Special tags are handled within Conary, and are the same regardless of namespace:

Creating Tags

There are two files per handler, both of which have to be implemented in the same component:

• Tag Description: A text file describing what files the tag should be applied to. This file should be placed in the /etc/conary/tags subdirectory (macro equivalent is %(tagdescriptiondir)s); examine the contents of this subdirectory for examples.
• Tag Handler: A script or executable that processes the tag actions. This file should be placed in the /usr/libexec/conary/tags subdirectory (macro equivalent is %(taghandlerdir)s); examine the contents of this subdirectory for examples.

Writing Tag Descriptions

A tag description resembles the following:

file          %(taghandlerdir)s/<name-of-tag-handler>
name          <short text to use in log.debug message>
description   <Arbitrary text>
implements    <calling convention>
datasource    args|stdin
exclude       <filterexp>
include       <filterexp>

• file specifies the filename of the script or executable, located in /usr/libexec/conary/tags
• name is optional, and defaults to the name of the tag description file. This is typically omitted.
• The calling convention argument to implements is one of files update, files preupdate (in Conary 1.0.26 and later), files preremove, files remove, handler update, or handler preremove; other conventions result in errors.
• datasource is optional, and defaults to args, which means that the filenames are passed to the tag handler as command-line arguments. stdin means that they are passed via standard input, one filename per line, which is useful for (rare) cases where a tag might be used for thousands of files. multitag means that they are passed via standard input, in pairs of lines, where the first line of the pair contains one or more tags, and the second line contains the filename that the tag or tags apply to:

tag1 tag2 ...
/path/to/file
tag2 tag 3 ...
/path/to/other/file

Each tag (here, tag1, tag2, and tag3) must have its own tag description file which lists the tag handler and specifies the multitag datasource. The multitag datasource protocol is useful mainly because Conary enforces no ordering between tag handlers, and so whenever there is an ordering constraint on handling multiple tags, they must be handled by the same tag handler.

• exclude and include : You can have arbitrarily many include and exclude lines. They are processed in order, and the first match wins. No match implies exclusion -- the files act as if the last line is "exclude .*" A packagepolicy will read the descriptions and apply them to each file packaged. You'll be able to override the results of applying tag handler exclude/include lines in a recipe -- though it would generally be preferable to fix the tag handler.

A <filterexp> is a regular expression as extended by filter.Filter. That is, if it starts with a / character, a ^ character is prepended to the expression so that it only matches from the beginning of a path, and if it does not end with a / or $ character, a $ character is appended to the expression so that the match stops there; "/lib/foo" matches neither /usr/lib/foo nor /lib/foo/bar. In other words, a <filterexp> is a regular expression that does what you obviously intend for matching paths.

A tag handler can call conary to query the conary database for any specific information it needs in order to do its work, though it cannot perform any operations that might change the database.

The following calling convention is used:

• $0 files update <filelist> This announces that the files in <filelist> have been either installed or changed, and lets the handler do whatever it needs to. Remember that this is called only for files that have actually changed name or content, not at every random package upgrade (unlike RPM). It is also not called if handler update is called, since the action of handler update is a superset of files update: it provides all the tagged files, not just changed tagged files. It is also called the first time a taghandler is installed, if that taghandler provides files update but not handler update.
• $0 files preupdate <filelist> This will be called before files are updated, only if the tag handler is installed on the system before that update job chunk is started. Note that files update will also be called. In general, this should be used only for circumstances that cannot be handled with files update, and if there is any chance that the taghandler will be installed concurrently with the tagged files, this tag handler may not be executed on initial install, but the tag handler must be written so that it will not break if it is executed on initial install. (Introduced in Conary 1.0.26.)
• $0 files preremove <filelist> This will be called before files are removed. Use only when necessary, such as with "service foo stop", where the file needs to still exist, or if the taghandler is in the same component as the tagged file. Do everything possible in the remove action instead. By the same logic as handler update, files preremove is not called if handler preremove is called.
• $0 files remove <filelist> This announces that the files in <filelist> no longer exist on the system, and requests an update. This will also be called for the old name if a file changes name. By the same logic as handler update, files remove is not called if handler preremove is called. Note that if the tagged file and the tag handler are in the same package, files remove will not be called if the package is erased, because the tag handler will have been erased; use files preremove when tag handlers and tagged files are in the same component and a removal action is required.
• $0 handler update <filelist> This announces that either the tagdescription file or the taghandler itself has been either installed or changed. <filelist> is a list of all files currently on the system carrying this tag. handler update can be particularly useful when a new version of a taghandlers has to clean up after bugs. Again, called on real change, not every time a package is installed; and if handler update is called, then files update is not called.
• $0 handler preremove <filelist> This announces that the tag description and tag handler files are about to be removed. (Obviously, we can't call the taghandler after removing it!) <filelist> is a list of all files currently on the system carrying this tag, facilitating cleanup. Note that if handler preremove is called, then files handler/remove is not called.

Each item of the calling convention which a tag handler supports will be

implements   files update
implements   files remove

Writing Tag Handlers

A tag handler in Conary is an executable which performs specific operations on a given file or list of files each time those files are changed. Appliance builders will occasionally use a tag handler to automate tasks based on file updates.

Tag handlers are only called for files that have actually changed from one version to another. The tag handler will be run after any tagged files have been changed on the system by applying Conary changesets (or updates). Tag handlers will not be called on an update if no tagged files have been changed; even if a package or group is updated, a tagged file itself must have changed for the tag handler to run.

Keep tag handler use to a minimum! Throughout this page, note the potential issues. Only use tag handlers when alternatives present a higher cost or a higher risk to system health.

Tag Handler API with Conary

A tag handler is passed a type such as files or handler as the first argument and an action such as update or preremove as the second argument. These types and actions are registered in the tag description. This makes error handling straightforward using case statements. See the two-level case statements in the tags in /usr/libexec/conary/tags/; rPath recommends using the structure of existing tag handlers as a guide to creating new ones.

The succeeding arguments passed to the tag handler are the paths of the tagged files that have changed. Recall that even if a package which contains tagged files is updated, the tag handler will not run unless at least one of the actual tagged files have changed.

File List as Arguments

In some cases, the list of files is not important to the tag handler; knowing that one of the tagged files was changed is sufficient to run the tag handler. This is common for database entries and cache entries in which the action is always a rescan of the supported paths. As an alternative, an update and remove could possibly be used to perform the same task.

In other cases, the list of files is important to the tag handler, and the list can be passed as the remaining arguments to the tag handler. In such cases, assume that more than one file can be passed because more than one tagged file may be changed in any given operation.

Multiple Packages and Package Versions

If a tag handler is expected to manage multiple versions of a package or multiple packages, it may not be called on one particular set of files or one version of a package at a time. The tag handler is run at the end of change operations, meaning it is possible that multiple items are updated at the same time.

Ordering

Conary does not provide ordering between tag handlers, and the order in which they are run is not guaranteed. A tag handler cannot expect that any other tag handler has run before it processes. This means that the runtime requirements for a given tag handler may have yet to run their own tag handlers. To handle things in a certain order, you will need to use multitag protocol handlers, in which one handler acts on more than one type of tag.

Tag Handler Updates

If a tag handler is updated, it is re-run against the tagged files on the system. This provides a means to correct errors in a tag handler, but it can expose new errors if you do not expect this behavior. For example, you may not want a tag handler to just append paths to a config file, but to also check to see if the file exists first because it is possible that the tag handler has been called on the same files before.

Shell for Tag Handlers

Though there is not specific requirement that a tag handler be written in shell, it is a best practice to ensure that dependencies are kept to a minimum.

Shell script runtime requirements are not automatically discovered, and when these dependencies are not present, the results are unpredictable. External dependencies required by the tag handler itself must be discovered and added as runtime requirements for the trove which contains the tag handler, and this is not done automatically for shell scripts. The runtime requirements must be worked out manually and assigned in the recipe. As a result, the more complex the tag handler, the greater the chance for missing runtime requirements.

Consider writing tag handler that are explicitly bash scripts and use bash conventions. For example, use ${path/*\//} instead of $(basename $path). Use the Parameter Expansion page as a reference.

Sample Tag Handler

The following example is the initscript tag handler with some additional comments to explain what it will do. Also, create a corresponding tag description file to accompany the tag handler.

#!/bin/bash
 
# Use at least enough arguments to ensure a valid type and action, or error out.
if [ $# -lt 2 ]; then
    echo "not enough arguments: $0 $*" >&2
    exit 1
fi
 
# Initialize type and action to leave the remaining args as the filelist.
type="$1"
shift
action="$1"
shift
 
# Use case type to ensure that the taghandler is passed only types it can
# handle.  In this case, use only handle files and error out on anything
# else
case "$type" in
    files)
        # For file type operations, case the possible actions and error
        # for anything unsupported.
        case "$action" in
            update)
                # Iterate the file list because it is not known in advance
                # how many files we will be passed in a given operation.
                for file in $@; do
                   /sbin/install_initd $file
                done
                ;;
            # Use preremove to cleanly stop a service
            # before removing the init file which controls it.
            preremove)
                for file in $@; do
                   $file stop
                   /sbin/remove_initd $file
                done
                ;;
            *)
                echo "ERROR: taghandler $0 invoked for an action ($action) that is not handled" 1>&2
                exit 1
                ;;
        esac
        ;;
    *)
        echo "ERROR: taghandler $0 invoked for a type ($type) that is not handled" 1>&2
        exit 1
        ;;
esac
 
exit 0