Conary:Group Recipe

From rPath Wiki

A group in Conary is defined by a group recipe. Like other recipe files, the filename is the group name followed by a .recipe extension. Conary also requires group names to be preceded by group-; a group recipe file for a group named group-example would be group-example.recipe.

Basic steps for developing the group recipe are:

1. Identify and locate the packages needed to build the group.
2. Write the group recipe to add the packages to the group.
3. Cook the recipe in the build environment and handle the Conary messages (cvc cook group-example.recipe).
4. Commit the group recipe to the repository and cook the group.

Use the following sections to create and develop a group recipe. As with all recipes, be sure to adhere to Conary's recipe conventions.

Group Hierarchy

Some packagers may want to create a hierarchy of groups defining an installed system, such as when developing appliances. Each group may be created with a separate group recipe and assembled in another recipe defining the system. Another approach is to write a single group recipe that includes the creation of its subgroups. Use the startGroup action to start new subgroups, and note that until the next startGroup, all lines will apply to that group:

    def setup(r):
        r.startGroup('group-sql-server', parentGroup='group-smf')
        r.add('mysql')
        r.add('mysql-server')

The startGroup definition exists in group-appliance and group-dist from rPath Linux and its derivative distributions. Most appliance builders will be building with those groups. The definition is as follows and can be used in recipes for groups that will not be built with group-appliance or group-dist:

def startGroup(r, name, parentGroup, depCheck=False, byDefault=True):
        r.setDefaultGroup(parentGroup)
        r.createGroup(name, autoResolve=autoResolve, depCheck=depCheck, byDefault=True)
        r.addNewGroup(name, groupName=parentGroup, byDefault=byDefault)
        r.setDefaultGroup(name)

Group Recipe Redirect

A Conary group redirect recipe is a way to point existing users to a completely different set of groups on a (possibly) different label. It can be used for major version updates (for example: updates.example.com@ex:product-3 -> updates.example.com@ex:product-4).

The following example assumes multiple groups making up one appliance. The top-level group is group-product-dist, which in turn contains group-product and group-product-core. All three groups are then redirected to the same group name on the new label: updates.example.com@ex:product-4:

 
class GroupProductDist(RedirectRecipe):
    name = 'group-product-dist'
    version = '0'
 
    def setup(r):
        for x in ['group-product', 'group-product-core', 'group-product-dist']:
            r.addRedirect(x, 'updates.example.com@ex:product-4', fromTrove = x)

In this example, any user updating a system currently on @ex:product-3 will be automatically redirected to the new groups on the new label when upgrading or installing.

If flavors are also different between the source and target of the redirect, such as redirecting from a vmware-flavored group to a non-vmware-flavored group, also add arguments sourceFlavor='vmware is:x86', targetFlavor='~!vmware is:x86'.

Group Scripts

Starting with version 1.1.20, Conary includes the group script feature for group recipes to run scripts when Conary install, update, and rollback commands are performed on the group. This feature is available for groups only (not for packages). The following shows example syntax for including group scripts in a group recipe:

class GroupExample(GroupRecipe)
    name = 'group-example'
 
    def setup(r):
        r.addPreUpdateScript('preupdate_script')
        r.setCompatibilityClass(2, groupName = 'group-example')
        r.addPostInstallScript(contents = '''#!/bin/bash
echo now > /tmp/state
''', groupName = 'group-example')
        r.addPostUpdateScript(contents = '''#!/bin/bash
echo now > /tmp/state
''')
        r.addPostRollbackScript(contents = '''#!/bin/bash
echo now > /tmp/state
''', toClass = 1)

Methods

Developers can use one of each kind of script per group: one for the group created by the group recipe, and one for each additional group created in the group recipe. Use the following methods to include group scripts:

Script Syntax

From the beginning, group scripts could use Python's multiple-line quote syntax (triple single quotes before and after) to allow the script contents to span multiple lines:

r.addPreUpdateScript(contents = '''#!/bin/bash
echo now > /tmp/state
''')

Be aware that the syntax between the triple single quotes is independent of Python formatting and should reflect the desired contents of the script. For example, there should be no newline after the opening triple single quotes and before the line defining the shell interpreter ("shebang"). Also, Python's normal indentation is not necessary between the triple single quotes.

Starting in a later version of Conary (since 1.1.20), group scripts could also accept a text file that is packaged alongside the group itself as part of the group's source. No other "r.add" line is needed in the group recipe since the script is not actually being installed as a part of the appliance update. The group script method call itself will look for that file in the group source and incorporate it as appropriate when building the group.

$> ls
CONARY        group-example-appliance.recipe        preupdate_script_file
$> cat group-example-appliance.recipe
loadSuperClass(group-appliance=conary.rpath.com@rpl:2)
class GroupExampleAppliance(ApplianceGroupRecipe)
    name = 'group-example-appliance'
    version = '1.2.1'

    def addPackages(r):
        # Pre-update script from a file
        r.addPreUpdateScript('preupdate_script_file')
        # Plus other packages to add to this appliance group

The script file that is part of the group source does not have to be executable. The text within the file is incorporated into a larger executable part of the group when the group script is run upon building.

In the case of the pre-update script, the group will first install a trove on the system before the update so that it can run the script from that trove. Then, the pre-update script is run, and the normal update to the group itself runs after the pre-update script exits successfully.

Group Name

Use groupName, optional in each function, to attach the script to an additional group defined in the recipe, such as a sub-group or related group. If a groupName value is not specified, the script is attached to the primary group associated with the recipe.

r.addPostInstallScript(contents = <script>, groupName = 'group-example')

Compatibility Classes

Compatibility classes can be set in a group to instruct Conary what rollbacks a group should be able to perform. If no compatibility class is specified, it is assumed to be zero (0). Because of this assumption, all groups created without setCompatibilityClass are in compatibility class zero.

Use setCompatibilityClass with the compatibility class value that should be associated with this version of the group recipe.

Use the toClass optional argument for addPostRollbackScript to declare which compatibility classes in previous group versions can keep their rollbacks available if they are updated to this group version. If the group that is updated to this version is of a compatibility class not specified in this version, the rollbacks will be invalidated, making them no longer available for Conary (or rAP) rollback operations.

For example, suppose this group version is set to compatibility class 2 and has a post rollback script with toClass with the single value of 1:

        r.setCompatibilityClass(2, groupName = 'group-example')
        r.addPostRollbackScript(contents=<script>, toClass = 1)

The following results would occur if groups of compatibility classes 0 and 1 update to this group version:

• If the group is updated from compatibility class 0, the rollbacks would be invalidated and no longer available. Rollback data is preserved even though rollbacks are unavailable.
• If the group is updated from compatibility class 1, the rollbacks would be available.

Scripts are run regardless of the compatibility classes defined. Developers should use the environment variables to determine if compatibility class has changed.

If the post-rollback script handles multiple compatibility classes, the toClass argument can be a list of integers.

Execution Process

Conary writes all scripts to its temporary directory (set to /tmp) and executes them from there.

Script Actions During Updates

The pre update script always runs before any update that includes the group. Pre update scripts run in an undetermined order, though all pre update scripts attached to a group will execute before those attached to its subgroups.

Only one of the post scripts runs, based on the action taken in Conary (install, update, or rollback). The database is available for Conary query operations while pre and post scripts are running.

Conary's update sequence is as follows:

1. Run the pre update scripts for the target group version
   Example: When updating from version 1 to version 2, run the pre update scripts for version 2.
2. Update individual packages and subgroups under the group to match the target group version
   Example: When updating from version 1 to version 2, update the installed components managed by group version 1 to match group version 2 (including adding any components that were not in version 1 or removing any components that are not in version 2)
3. Update the group itself to the target version
   Example: When updating from version 1 to version 2, change the installed version information to version 2.
4. Run the post update scripts from the target version
   Example: When updating from version 1 to version 2, run the post update scripts for version 2.

In Conary 2.0.7 and earlier, no pre or post update scripts are executed at the time of a rollback. In Conary 2.0.8 and later, when a group is rolled back to a previously installed version, Conary runs the pre and post update scripts for that earlier version, though in a different sequence than used during updates. Conary's rollback sequence in version 2.0.8 and later is as follows:

1. Run the pre-rollback scripts from the currently installed version
   Example: When rolling back to version 1 from version 2, run the pre-rollback scripts from version 2.
2. Run the pre-update scripts from the previously installed version
   Example: When rolling back to version 1 from version 2, run the pre-update scripts from version 1.
3. Update the group itself to the previously installed version
   Example: When rolling back to version 1 from version 2, change the installed version information back to version 1.
4. Run the post update scripts from the previously installed version
   Example: When rolling back to version 1 from version 2, run the post update scripts from version 1.
5. Update individual packages and subgroups under the group to match the previous group version
   Example: When rolling back to version 1 from version 2, ensure all the installed components managed by version 2 of the group are rolled back to the components specified in version 1 of the group (including are removing them if they do not exist in version 1, or adding them if they do not exist in version 2)
6. Run the post rollback scripts for the group version prior to the rollback
   Example: When rolling back to version 1 from version 2, run the version 2 rollback script

Reference how group scripts fit into the process of updating a group.

Conary group redirects are always considered new group installs, even if the group name is the same as a currently installed group.

Scripts are run only if they are provided in the update target. If an installed group does not have a script, but an update to the group includes one or more scripts, the new scripts are run as provided in the group recipe (including any pre update scripts). Likewise, if an installed group has scripts, but an update to the group does not, no scripts are run.

Points of Failure Considerations

Scripts must return an exit code of 0 to indicate success; any other value is interpreted as a failure. Consider these other points of failure:

• If an install fails for a package in the group, the entire update operation is aborted, and Conary takes no further actions. If a pre-update, pre-install or pre-rollback script is associated with a group that has not yet been updated (or rolled back), the pre-script will be re-executed when the update (or rollback) process is restarted.
• If a pre-script (pre-install, pre-update, pre-rollback) fails, the entire operation is aborted, and Conary takes no further actions. Since a pre-script can possibly run several times, the developer should take care to write pre-scripts in a manner that makes them robust when run multiple times.
• Failures in post scripts are ignored, though they are reported to the user.

The post-rollback script for a group is guaranteed to be run if at least part of the group was installed. If a group update fails after installing some of the packages, and the update operation is resumed, the rollback script may be executed multiple times. The reason for this is the fact that the script is probably trying to undo a certain action for a package that may have been installed already.

To deal with the potential for multiple execution, rollback scripts should be designed to be idempotent - that is, running them a second time should not result in negative effects. If the post-rollback scripts depend on the current version of the appliance, they should confirm that the appliance is currently at the state that they expect.

Environment Variables

Recall that a group is unique given its Conary name, version, and flavor. These environment variables are set during the run of all scripts:

• CONARY_NEW_NAME
• CONARY_NEW VERSION
• CONARY_NEW_FLAVOR
• CONARY_NEW_COMPATIBILITY_CLASS

These environment variables are set during the run of all scripts when an installed (old) group is updated or rolled back:

• CONARY_OLD VERSION
• CONARY_OLD_FLAVOR
• CONARY_OLD_COMPATIBILITY_CLASS

The PATH environment variable for all scripts is set to /usr/bin:/usr/sbin:/bin:/sbin. No additional environment variables are set unless they are set by the script itself or the script interpreter.

Conary Developer Notes

If a root is specified during an update (using --root), scripts are executed in a change root (chroot) if the process is running as root. Otherwise the scripts are silently ignored.

Use --tag-script <filename> with a conary update command to write all scripts to /tmp and to generate the tag script file <filename> to run them. When the option is used, the tag script file and group scripts are not run. Also, pre script invocations are commented out in the tag script as they are normally used for initial installs.

Pre script invocations should be self contained. A group script can be updated from any previous version (including absolutely nothing for a fresh install). Making assumptions about what libraries will be available leads to errors. A pre-update script failure prevents the group from being installed (see Points of Failure Considerations)