rPath Appliance Platform Agent:Backup Plugin

From rPath Wiki

The rPath Appliance Platform Agent (rAPA) includes a Backup plugin which is enabled by default. The plugin provides the Backup and Restore page in the rAPA web interface. The Backup plugin allows a rAPA user to launch a manual backup of the appliance, schedule regular appliance backups, or restore the appliance from a backup.

Using the Backup Plugin Interface

See the Backup and Restore page in the Appliance Administration pages for information on how to use the Backup plugin in the web interface.

Appliance Developer Notes

The following information should be referenced by appliance developers who are including the rPath Appliance Platform Agent in appliances.

Back Up and Restore Sequences

rAPA has a particular sequence in which it performs its backup and restore operations.

Backup operations use the following sequence:

1. Run each script in /etc/raa/backup.d/ with clean
2. Assemble a list of files to back up
3. Run each script in /etc/raa/backup.d/ with prebackup
4. Run each script in /etc/raa/backup.d/ with metadata
5. Run each script in /etc/raa/backup.d/ with backup
6. Mount the backup destination
7. Create the tar archive on the destination
8. Validate the tar archive
9. Unmount the backup destination
10. Run each script in /etc/raa/backup.d/ with postbackup

Restore operations use the following sequence:

1. Run each script in /etc/raa/backup.d/ with clean
2. Mount the location where backups are stored
3. Extract the backup metadata from the tar archive
4. Run each script in /etc/raa/backup.d/ with isValid, passing the metadata as a standard input stream (stdin)
5. Run each script in /etc/raa/backup.d/ with prerestore
6. If backup.validateSha1 is set, validate the SHA-1 digest of the backup archive
7. Extract the backup files
8. Run each script in /etc/raa/backup.d/ with restore
9. Run each script in /etc/raa/backup.d/ with postrestore"
10. Reboot the system

The scripts in /etc/raa/backup.d/ either need to accept a single argument, or it should require no argument and should just exit.

Configure the Backup Plugin

Appliance developers can configure the backup plugin for the specific needs of an appliance product. The default configuration is located in the file backup/backup.cfg in the rAPA plugins directory, but appliance developers should use a custom.cfg file as described in rPath Appliance Platform Agent:Configuration to override these defaults when necessary. Be sure to note the configuration section (in brackets) within [global] for each item described in the following sections, and be sure to use the same corresponding section within [global] in the custom.cfg file.

Though there are other directives in backup.cfg besides those described below, those directives are not supported for anything other than their default values.

Enabled and Disabled Backup Types

[/backup] -- backup.disabled_types

Use backup.disabled_types to disable whether the appliance use particular available backup types. The values for the directive correspond as follows with the interface options:

• Downloadable Backup File (HTTP) -- URL
• Network File System Share (NFS) -- NFS
• "Windows" File System Share (CIFS/SMB) -- CIFS
• Mounted Filesystem (Label) -- LABEL

By default, URL is disabled and all others are enabled as shown in this section from the backup.cfg file:

[/backup]
# Disabled backup types:  default is the empty list
# Available backup types are currently 'URL', 'CIFS', 'NFS', 'LABEL'
backup.disabled_types = ['URL']

To override this default, place the directive and its overriding value in the [/backup] section in custom.cfg. In the following example, custom.cfg disables CIFS and NFS, leaving only URL and LABEL enabled:

[global]

[/backup]
backup.disabled_types = [''CIFS'', ''NFS'']

Include supporting packages in the appliance to support the selected backup type. The backup plugin supports backup to multiple types of targets. The appropriate supporting software must be supplied with the appliance to ensure the backup and restore processes execute correctly. Though the NFS and CIFS/SMB types are included automatically with rAPA, the following should be considered requirements when using these backup types: NFS -- include the portmap and nfs-utils packages SMB/CIFS -- include the samba:runtime component

Temporary Directory

[/backup] -- backup.tmp_dir

Use backup.tmp_dir to identify a location on the appliance's filesystem for temporary files and on which to create mount points. By default, this value is set to /tmp as shown in this section from the backup.cfg file:

[/backup]
# Directory to treat as the temporary file location as well as the place to
# create mount points.
backup.tmp_dir = '/tmp'

To override this default, place the directive and its overriding value in the [/backup] section in custom.cfg.

Local Storage for the Download Type

[/backup] -- backup.local_storage

Use backup.local_storage to specify a location on the appliance's filesystem to store backups for the Downloadable Backup File backup type. This configuration is only relevant when URL is removed from backup.disabled_types (to enable the plugin, which is disabled by default). By default, the backup file that will be downloaded is created in /var/lib/raa/backups as shown in this section from the backup.cfg file:

[/backup]
# Location to store backups when using the DOWNLOAD
# backup type.
backup.local_storage = '/var/lib/raa/backups'

To override this default, place the directive and its overriding value in the [/backup] section in custom.cfg.

Location of File Lists and Scripts

[/backup] -- backup.file_list_dir

Use backup.file_list_dir to specify a location on the appliance's filesystem from which to read file lists and scripts used to customize the backup process. See the Identify What to Back Up section for more information on packaging alternate lists and scripts for the appliance. By default, the location used is /etc/raa/backup.d as shown in this section from the backup.cfg file:

[/backup]
# Location to iterate through reading file lists and
# running scripts
backup.file_list_dir = '/etc/raa/backup.d'

To override this default, place the directive and its overriding value in the [/backup] section in custom.cfg.

Backup Filename Format

[/backup] -- backup.file_format

Use backup.file_format to indicate the format of the filename for the backup. By default, the backup file has a name that users year, month, date, hour, minute, and time zone as shown in this section from the backup.cfg file:

[/backup]
# Format for the backup file. See man strftime for available substitutions
backup.file_format = 'backup-%Y%m%d-%H%M-%Z.tar'

An example of a file generated by this default filename is backup-20080310-1053-UTC.tar. To override this default, place the directive and its overriding value in the [/backup] section in custom.cfg. Use man strftime at a Linux command prompt to read the manual page that describes the accepted date and time variables that can be used in the filename.

Filter for Backups to Keep

[/backup] -- backup.filter_expression

Use backup.filter_expression to provide a regular expression for filtering backup files. By default, the filter uses the regular expression shown in this section from the backup.cfg file to indicate files that follow the default backup filename format with tar or tgz extensions:

[/backup]
# This is a regular expression that allows you to filter which available backups to
# keep in the system via the prune method.
backup.filter_expression = 'backup-.*\.(tar|tgz)'

To override this default, place the directive and its overriding value in the [/backup] section in custom.cfg. Be sure this change is compatible with the value used for backup.file_format as described in Backup Filename Format.

Validate SHA-1 for a Backup

[/backup] -- backup.validateSha1

Use backup.validateSha1 to indicate whether the validate a backup using its secure hash algorithm. rAPA generates a SHA-1 message digest for each backup that can be used for this validation. By default, validation is off with a value of False as shown in this section from the backup.cfg file:

[/backup]
# Verify a backup's SHA1 post-backup/pre-restore
# Turning this on will take lots of time for large backups over a network share.
backup.validateSha1 = False

To override this default, turning on validation, place the directive and its overriding value in the [/backup] section in custom.cfg. In the following example, custom.cfg enables validation with SHA-1:

[global]

[/backup]
backup.validateSha1 = True

Backup validation extends the time required for a backup or restore to occur.

Identify What to Back Up

Appliance developers can configure what data on the appliance should be included in a backup. Some types of data to consider for backup include:

• Application configuration files
• System configuration files
• Databases
• User data

Most of these items can be handled with file lists that specify the full path to each file. However, databases might require a database backup application or script that can connect to the database and draw the required data into files. These files' names should be passed as output when the database backup is executed. Package the appliance file lists and any backup scripts to install in /etc/raa/backup.d on the appliance.

rPath recommends that appliance developers back up only essential data that is not easily recreated. This ensures the smallest backup files possible. Avoid transient data such as temporary files and cache.

File Lists

Create file lists defining critical files to be backed up on the appliance. The list is a text file without executable bits set and containing a list of files or directories to back up, one item per line. If a developer lists a directory, the backup plugin will back up all content below that directory level.

Shell conventions such as brace expansion and globs may also be used in the filenames comprising a list. See the examples that follow.

The backup plugin includes directories above a specified directory or file, but only for creating the directory structure. For instance, if you back up /etc/example/config, the backup contains the directories /etc and /etc/example, but those directories are empty except for what is specified in the file list (in this case, just /etc/example/config). Select the directory levels appropriate for the appliance backup.

rAPA provides /etc/raa/backup.d/rAAFiles that specifies a certificate and key files:

/etc/raa/raa-service.{privkey,pubkey}
/etc/ssl/pem/raa.pem

rAPA also provides /etc/raa/backup.d/SuggestedFiles listing some system configuration files suggested for backup:

/etc/modprobe.conf
/etc/hosts
/etc/resolv.conf
/etc/localtime
/etc/sysconfig/{network,network-scripts/ifcfg-*,network-scripts/keys-*,clock,i18n,iptables,hwconf,firstboot,grub,keyboard,mouse}

To customize this selection, package an alternate directory to install on the appliance with the desired lists, and use custom.cfg to point to that directory as described in previous section Location of File Lists and Scripts.

Backup Scripts

Create backup scripts to run backup-related tasks. Write scripts to provide and accept file names when file lists are not sufficient, such as with database engines. In such cases, a script can connect to the database and output data to files which can be specified in a file list as script output.

Note that a script requires that the file's permissions include the execute bits in order to be recognized as an executable file. Otherwise, rAPA reads the file one of the file lists within the same directory.

rAPA provides /etc/raa/backup.d/rAADatabase, a Python script that performs a backup of the appliance's rAPA database. Examine this script as an example for composing similar scripts. Defined functions that you can reuse and adapt include:

• backup: instructs the script to perform its backup functions and print a list of files or directories to backup on standard output
• restore: instructs the script to perform its restore functions; the script must be configured to handle the appropriate files
  Example: your script can respond to all files in a specific temporary directory, such as /tmp/exampleBackup; the temporary data should be cleaned up when your script receives the clean argument described below
• clean -- instructs the script to clean any temporary data generated and used during backup or restore
• prebackup -- instructs the script to perform any necessary functions before backup starts
• postbackup -- instructs the script to perform any necessary functions after backup finishes
• prerestore -- instructs the script to perform any necessary functions before restore starts
• postrestore -- instructs the script to perform any necessary functions after restore finishes
• metadata -- instructs the script to output metadata (in key=value form, one key per line) to be stored in the backup's metadata section
• isValid -- instructs the script to read metadata from stdin and validate whether a backup can be restored to a particular appliance

Backup Plugin on the rPath Update Service

As of rPath Appliance Update Service 4.0, the rPath Appliance Platform Backup Plugin is installed and enabled by default. The plugin can be configured through the administration interface and is identical in operation to the standard backup plugin. Be advised that this backs up all data and creates a compressed archive locally on the rUS for retrieval. Depending on the size of the rUS, this archive may be quite large. Be cautious before attempting these backups.