Conary:Concepts
From rPath Wiki
|
The Conary software provisioning system is a means to package, deliver, and maintain software. Conary packaging technology is the open-source package management system at the heart of rPath's products and services.
Conary provides new features to package management, addressing some of the common issues of Linux system administration. Employing repositories, a branch structure with shadowing, and changesets, Conary offers the advantages of a revision control system to improve system maintenance tasks such as updating current software, installing new software, maintaining configurations during software updates, and resolving dependency issues. These features extend beyond previous software packaging technologies, creating a system in which many traditional Linux software maintenance tasks are automated and efficient, significantly reducing the time spent on system management.
Conary also offers a finer breakdown of system components and a structure designed to be easily streamlined. This streamlining can be used to build a dedicated software appliance in which the only installed pieces of the operating system are those necessary to support a particular application. rPath Linux is built using Conary packaging technology, and rBuilder is designed for developing and distributing Conary-based appliances.
Repository
A Conary repository is a database that contains source and binary files organized in packages which consist of any number of components. This database is accessible over a network (including over the Internet if needed) by Conary-based systems, and it is the means by which these systems can install and maintain their software. The repository contains multiple packages in multiple versions along multiple development branches, and Conary ensures that system maintenance uses the appropriate version on the appropriate branch when updating a system. Access to a package in the database requires enough information to distinguish the exact version required, but much of this information is maintained in configuration files and installed package information to further automate the update process.
Repository Contents
A repository provides the packages that can be installed on a Conary-based system. Other types of content in the repository include components, groups, and filesets, all of which represent a type of trove, or versioned item in the repository.
Packages
A package represents a piece of software that can be installed on a Conary-based system. When software is packaged for Conary, building the package creates components, and the package contains those components. The Conary QuickReference provides instructions for installing, updating, and removing packages on a Conary-based system.
When a user installs a package, Conary installs all the components of that package. Conary will also indicate any dependencies the package requires, and the user can add an option to let Conary automatically resolve those dependencies. When Conary installs a package with this option, it additionally installs only the components necessary to resolve the dependency.
When a user updates installed packages, Conary only retrieves files that have changed since the last update on the package. For example, if the component foo:runtime has five files, and only one of those files has changed between two versions, then only that one file will be updated when updating between the two versions.
From the respository's perspective, Conary aims for similar efficiency. The new version of a package does not contain duplicates of files in previous versions. Only if a file has changed will its update information be saved to the repository.
Not all Conary users will need to package software, but those that do may find it helpful to understand Conary packaging concepts prior to packaging. Packaging software for Conary includes:
- Writing a recipe to tell Conary how to build the package
- Cooking the recipe and testing the package repeatedly in an effort to develop the recipe
- Committing the package to a repository and cooking the package in the repository
The Conary New Package Tutorial introduces this packaging process to both new packagers and experienced Linux packagers who are new to Conary.
Components
A component is a grouping of files that are needed to install some specific part of software, such as the application, libraries, or documentation. When a package is built, Conary automatically assigns files to components (though this assignment can be overridden if desired). Components are identified using the package name to which it applies, a colon (:), and a suffix describing the component (such as package_name:lib and package_name:docs).
A source component (:source) contains files and is named like other components, but Conary treats source components differently during development and maintenance of packages. Though it is named using the original package for which it was built, a source component is versioned differently, and it is not considered part of any package. A source component may be used to build multiple packages, and it not installable like other components.
Troves
A trove is a collection of files or a collection other troves. The term trove is used often in Conary discussions. A trove could refer to a component, a package, a group, or a fileset.
A trove is identified by name, version, and flavor; these three values make the trove unique in the repository. When a file in a trove is first saved to a repository, it is assigned a unique identifier so that it can be tracked by that identifier instead of by name. The file identifier allows Conary to handle continued updates to the file, including filename changes, while preserving its unique existence in the trove.
Groups
A group contains a grouping of other kinds of troves (components, packages, filesets, or groups) that are found in the same repository. Groups are mainly used to pull together packages and components that should be installed together to create a complete infrastructure for software. A group name has the prefix group-. The Conary QuickReference provides instructions for installing and updating groups on a Conary-based system.
Groups are versioned like other troves and can be updated in the same way. The advantage of creating a group of packages and components is that these troves can be updated and maintained together. The group identifies specific versions of each of its contents. Installing a group means installing those specific versions of packages and components. Because of this, installing or updating a group may not result in receiving the latest available version of its contents because a package or component in the group may have a later version available outside of the group. Instead, installing and updating by group ensures that the group maintains the versions that were intended. Groups allow a developer to distribute software that is assured to work together. If packages or components in a group are updated, the group update will only affect those updated troves.
For example, suppose you have installed group-foobar on your Conary-based system which consists of packages foo-1.2.1 and bar-2.1.1. Suppose you are told that newer version foo-2.0.1 is now available, and you perform a system-wide update on your system to ensure you have the latest version. Because foo was installed as part of the group and not on its own, your version of foo remains 1.2.1. Then, suppose group-foobar is updated to use bar-2.2.1. Another system update results in a group update; while foo remains in version 1.2.1, bar is updated to version 2.2.1.
Filesets
A fileset contains only files from components in the repository. Filesets can be created to designate a custom group of files in the repository outside of the component, package, and group purposes. A fileset name has the prefix fileset-. This type of trove is rarely used outside of low-level Conary development.
Repository Access
Conary uses client-side repository access features that enhance Conary's functionality.
Repository Map
A repository map setting on a Conary-based system maps a hostname from a label to a full URL for a networked repository. When software is installed using Conary, the label is part of the information that identifies the software and its source. If the repository changes locations, the repository map can be updated to ensure that Conary finds the relocated repository when a user runs software updates. Repository map settings are maintained in a Conary configuration file on the Conary-based system.
Install Label Path
Though it is an option, Conary does not require a user to specify a label when installing a trove from a repository. When a user names the trove (with or without specifying a revision), the repositories listed in the system's install label path are searched in order to locate that trove and fulfill the request. As needed, repositories can be appended to or removed from this path. Install label path settings are maintained in the installLabelPath line of a Conary configuration file on the Conary-based system.
Mirrors
Conary provides the option to mirror a single repository onto another Conary system. This can be useful when backing up a repository or when making privately-developed software packages available in a publically accessible repository. Conary users can create a standalone repository and mirror to that repository. Additionally, project owners on an rBuilder Online (rBO) project can create a mirror of the rBO repository identified with that project. On the software appliance scale, appliance developers and maintainers can use the rPath Appliance Platform Update Service appliance to manage mirrors provided to multiple consumers (such as software vendors or customers).
Label Structure
Conary uses labels to simplify update paths for software. This structure is aimed at eliminating the guesswork when identifying:
- Whether the package is "newer" or "older"
- The appropriate upstream development path the current package has followed to its current version
Conary identifies each component with a version string that identifies the originating respository and revision number. When a user performs system update activities, Conary uses this string as the update path to follow for each trove.
Releated page: Conary:Version String
Label Affinity
Conary supports divergent development by allowing that development to occur on different branches. Such development in the Conary repository creates a branched structure and a need to identify which label to use when updating packages installed from the repository. When a trove is installed on a Conary-based system, the updates for that trove will come from the same label in the respository from which it was installed. This concept, called label affinity, ensures the update path is consistent, even when the development is not linear. The identifiers that Conary creates for ensuring label affinity are hidden from users, and it is not necessary to know them when updating software on a Conary-based system.
Conary recognizes an appended version string with a slash and labels as a unique identifier for a development branch. When the branch is created, the branch name becomes the label for that branch. This label inherits the namespace, so it can be referred to not only with the longer slash-separated strings, but also in place of the original tag. In the example, 1 is the branch name, and the full label for the branch is conary.rpath.com@rpl:1.
/conary.rpath.com@rpl:devel//1/1.5.0-2-3 or conary.rpath.com@rpl:1
If a package is installed from this label, the label name ensures that Conary uses the correct update path in the repository.
Historically, Conary had separate branch and shadow creations. Explicit branch creation was deprecated so that shadows alone create a repository's branch structure.
Versions
As previously described, Conary structures packaged software in components, and each component contains one or more files. The version string provides a globally unique identifier for the component. See Conary Version String which describes the version string and its parts, including the commonly used branch, label, and revision values.
Flavors
Along with the name and version, the flavor of a trove makes it unique among troves that can be installed on a Conary-based system. In the same way that the version identifies where and when in a sequence the trove was built, the flavor identifies how it was built. This information is part of the build instructions and includes selecting options in the packaged software, such as designating an architecture or ensuring support for a particular database. Flavors are typically invisible to users installing software because Conary automatically selects the most appropriate flavor for a package for the target system. Conary provides an option to override this automatic choice to users wishing to select a particular flavor of the package.
Related page: Conary:Flavors
Shadows
Conary provides the shadow concept as a means of creating parallel development branches. Shadows can keep track of upstream changes from a parent branch, and developers can choose to merge changes from any new revision on the parent at any time. In this way, developers can be responsible for their own changes without maintaining the entire development stream.
Related pages: Conary:Shadows | Conary:Merge
Changesets
A Conary changeset is a file that describes modifications to be made to a system in order to apply a Conary update. A changeset can include how particular files have changed as well as how their associated troves have changed. Conary uses changesets to make the update process more efficient and less intrusive to a Conary-based system. By applying only the changes that are needed, files that have not changed are not needlessly updated, and locally-changed files are merged with updated files instead of overwritten.
A changeset represents the difference between two versions. An absolute changeset is the difference between having no version installed on a system and installing the desired version. A relative changeset is the difference between a currently installed version and the desired version. A relative changeset could imply either an upgrade to a newer version or a downgrade to an older version.
Changesets are typically invisible to the user making updates to a Conary-based system. During an update operation, Conary creates the changeset dynamically before downloading and applying that changeset. The changeset may be cached by the repository server, but it is not saved as a separate file on the target system. In some instances, a system administrator or software developer may generate a static changeset file to distribute a desired set of changes.
One use of a static changeset is a local changeset, which is a relative changeset containing the differences between the trove as installed and configured on the local system and the trove as it exists in the repository. System administrators can distribute certain configuration and permission changes on a trove by creating and distributing a local changeset. The local changeset can be distributed to target machines with the same trove version, and it can be committed to a branch of the repository from which target machines can install the necessary update.
Rollbacks
As a result of the use of changesets in Conary, a Conary-based system provides the user an opportunity to roll back changes that have been made to the system as the result of a Conary update. Conary maintains a list of rollbacks, and the system administrator can roll back a trove or the entire system when necessary to bring the system back to a particular state.