Appliance Development:Linux Software Packaging History and Concepts

From rPath Wiki

  APPLIANCE DEVELOPMENT

Extensive resources have been published about Linux history, administration, software development, and operating system development. This page does not attempt to summarize or substitute those resources, but to offer a brief view of Linux operating system and packaging structure for new Conary users who may be new to developing software for Linux. For more details about what is outlined here, consult any of various resources available online and in printed publications.

The Linux Operating System

Linux is made up of four primary layers:

• Kernel/drivers that provide hardware interaction, data access, and job scheduling
• Libraries that provide shared functionality for layered applications
• Shells that provide command line interaction and execution for applications
• Layered applications that interact with a shell or the kernel (including core system software)

Kernel

Linux uses a monolithic kernel which runs in a kernel space in supervisor mode. The kernel can be extended with modules that dynamically load at runtime. Modules allow the operating system to function on various combinations of hardware and provide users with flexibility to create, add, and remove modules to extend the kernel as desired.

All drivers that interact with hardware must either be loadable modules or integrated components of the kernel. The Linux kernel does not generally include binary-only drivers, which would be closed-source.

Building a custom kernel and modules represents early efforts in appliance building, streamlining a Linux system to only load and use what is necessary to run particular software on particular hardware.

Shells

A Linux shell is a command interpreter that provides access to the kernel operations. Different shells use different command structures. Common command line shells fall into the C shell and Bourne shell families. Graphical shells include the X Windows System as well as window managers and desktop environments based on X.

Linux software runs in one or more shells and references runtime libraries, such as GNU's C library glibc, that provide a binary interface between the software an kernel.

Shell scripts run shell commands in a certain type of command line shell, and they can be used to direct other Linux software.

Filesystem

All elements of a Linux install are files. Each file exists in a filesystem, which is a way that a disk partition is formatted allowing the kernel a structured way to read and write data from the physical media. A filesystem in Linux is built as a descending tree of directories starting with the root directory, referred to by its slash symbol: /

The directories under the root directory represent the different types of files that make up software. Applications install files into these directories, which are consistent or very similar across Linux distributions. For examples, configuration files that define how an application is expected to run will typically be in /etc (pronounced "et-see"), documentation files will typically be in /usr/share/doc, and binary application files will be in /usr/bin, /sbin, or a similar directory.

Distributions

A Linux distribution (distro) is comprised of the Linux kernel, all or part of the GNU operating system, and assorted other software. Distributions may be commercially driven, such as rPath Linux (rPath), Red Hat Enterprise Linux (Red Hat), and SUSE Linux (Novell); or they can be community-driven, such as Foresight Linux, Debian, and Gentoo. Most distributions are segmented into packages, and distro developers package software for the distribution using whichever package management is used by the distribution.

Packaging Software for Linux

A package includes all components of a particular computer program. A package can also dynamically set configuration based on the system environment on which it is installed.

Without a package to define the installation of an application, one must install each individual file manually in the appropriate location and configure them correctly, including setting permissions and ownership for each file.

Packages can be created to use the operations in other packages. A dependency is a component contained in one package that is required by another package. This means that one package depends on the other. Each packaging in a dependency relationship is said to provide or require certain dependencies. A package that requires one dependency must be installed with another package that provides that dependency.

Package Management

Package management provides a means to install and maintain packages on the system while handling dependencies between packages.

Archived+Compressed

Archiving technology and compression technology can be used together to gather files into a single package file. Dependencies are handled manually. No logic is built in to the package to control the installation based on the local environment.

To install such a package, one unpacks the files and follows package-specific instructions to make and install. A popular form of these packages is tar and gzip combined to create a .tar.gz or tgz file, often called a tarball.

Managed Packaging

Managed packaging combines a custom file type and a package manager that uses that file type for packaging files. Most or all dependencies for packages are handled by the package manager.

To install such a package, one uses commands specific to that package manager. Popular forms of managed packaging include RPM (Red Hat, Fedora), dpkg (Debian, Ubuntu), Portage (Gentoo), and rPath's Conary.

rPath co-founder Erik Troan was the original author of the RPM Package Manager, formerly the Red Hat Package Manager and currently the most widely-used form of managed packaging. Erik went on to develop Conary with Michael K. Johnson and Matt Wilson.

Recipe or specification (spec) files provide the package manager instructions to create the package. For most managed packaging, the packager must provide specifications for dependencies, which can lead to incorrect or unnecessary dependencies. Also, most managed packaging requires the entire required package when a dependency is specified, which can cause a dependency set to grow rapidly.

Repository-based Package Management

Repository-based package management provides a network-accessible store called a repository from which packages are available for installing and updating. Conary package management is repository-based.

Some package managers allow systems to install from multiple repositories, and some repositories use a channel to define a distribution.

Besides providing a more direct way to download packages, some repositories have expanded to track the package manifests of individual deployed systems, as with RedHat Network, or to dynamically assemble derivative operating system builds, as with Conary.

Conary

Conary was built to replace existing package management systems. rPath technologies such as rBuilder and appliances created using rBuilder are based on Conary package management.

Contributions to this page are encouraged, especially from those with varying experiences and insights to offer on these subjects. Log in and EDIT the page directly, or provide comments by visiting and registering for the rPath Forum