From rPath Wiki

An Overview of rPath Appliance Platform Plugin Development

Appliance developers can create custom rPath Appliance Platform Agent plugins to perform additional administrative tasks beyond those in the standard plugins. For example, an intrusion detection appliance may require a custom plugin to process detection rules, or a web server appliance may need a plugin to add virtual host entries to its configuration.

The following pages provide reference and instruction for plugin development. The list is ordered to provide structure for first-time developers:

1. Reference: Plugin Structure
2. Establish a plugin development environment.
3. Add one or more example plugins for reference.
4. Lay out the plugin appearance in the web interface.
   1. Reference the kid templates used for content layout and presentation.
5. Step through the plugin development process. (in development)
   1. API for Plugin Development
   2. Create and schedule plugin tasks.
   3. Reuse code from rAPA and functions from other plugins.
   4. Use Python decorators in the plugin code to expose methods by XML-RPC, JSON, and other means.
   5. Ensure the plugin can be used by XML-RPC clients
   6. Create plugin database tables.
   7. Add static files used by the plugin.
   8. Designate user roles and corresponding functions for a plugin.
   9. Troubleshoot errors when developing and using plugins
6. Package the custom plugin.
7. Configure plugins and override appliance agent defaults.
8. Configure plugins to be part of the Configuration Wizard.

rPath Appliance Platform Agent was previously branded as rPath Appliance Agent, abbreviated "rAA". Many files, directories, classes, and methods will have the "raa" string adapted from the agent name that drives the Appliance Platform.

Plugin Structure

Plugin Structure

A plugin in the rAPA consists of Python code and related files used to accomplish a particular administrative task. Plugin files reside in the directories associated with the agent or in an alternate location configured for searching plugins. The agent finds plugins at these configured locations and interprets the code to render the plugin interface and execute its underlying functions.

A plugin is referenced in configuration files by a plugin identifier. The identifier has the format /plugin/PluginName where plugin is the directory in which the plugin files reside and PluginName is the name of the Python class that defines the plugin. Standard appliance agent plugins reside in /usr/lib/raa/raaplugins/ by default.

The action of a plugin is driven by tasks. The two components of the plugin each handle different tasks:

In the Python code used for plugins, developers use methods (functions) associated with a class to pull in different rPath Appliance Platform Agent operations. Methods in the web component can correspond to a URL so that when the browser navigates to that URL, the method is run and its return value is sent to the browser. For example, if the plugin is project.plug.web.plugin.Plugin, then the index method is evaluated when the user browses to /rAA/plugin/PluginName/, and the example method is evaluated upon the request of /rAA/plugin/PluginName/example.

The web component can request two types of task execution by the service component:

• Immediate execution is for tasks which complete quickly (such as changing the root password), but require root privileges. For such tasks, the call for execution made by the web component must wait for the service component to complete the task.
• Scheduled execution is for tasks which may run longer than a few seconds, whether or not they require root privileges to execute.

The following images show rPath Appliance Platform Agent infrastructure and plugin structure:

rAPA Infrastructure

rAPA Plugin Structure

Creating the Plugin Development Environment

Establish an environment for rPath Appliance Platform plugin development by installing rPath Appliance Platform Agent, creating directories for development, and changing the software to development mode. This should be separate from any Conary-based appliance development (Conary Build Environment), though the resulting plugin should be packaged in Conary and added to the groups defining an appliance.

Install Software

Conary-based systems may already have rPath Appliance Platform Agent installed. Query the system to determine the version installed:

$> conary q rapa
rapa=3.0-1-0.1

If the package is not installed, install group-rapa for use in plugin development:

#> conary update group-rapa=raa.rpath.org@rpath:rapa-3

Create Directories for Development Work

Create directories for plugin development as described in the following steps:

1. Create a directory for plugin development, such as ~/appliance-plugins.
2. Create a directory for each rBuilder product on which you are developing plugins.
3. In each product directory, create a directory for each plugin you are developing.
4. For each plugin, create a web directory to correspond to the web component and srv directory to correspond to the service component. You can specify any name for the component files, but the Python class name of the web component must match the Python class name of the corresponding service component.
5. Create an instance of the file __init__.py in each directory comprising your plugin's directory structure. This file can be empty, perhaps created with touch, but it must exist to identify the directory as a Python module.

In summary, the directory structure in your development directory should resemble the following:

• <development_directory>/<product_name>/<plugin_name>
• <development_directory>/<product_name>/<plugin_name>/srv
• <development_directory>/<product_name>/<plugin_name>/web

These directories are part of the Python namespace and should contain only characters allowed in Python namespaces.

Change to Development Mode

The rPath Appliance Platform Agent is distributed with a tarball containing files to enable development mode operations. In development mode, messages are output to standard output instead of logged to files.

Before changing to development mode, be sure the appliance agent is not running. Use the following command to check whether raa is a running process:

$> ps auxwww | grep raa

If necessary, stop the appliance agent by stopping the raa and raa-lighttpd services:

#> service raa stop

To configure development mode, unpack the rPath Appliance Platform Agent development files and make modifications as described on this page.

Change to the development directory and unpack /usr/share/doc/rapa-3.0/devel.tar.bz2 to it:

$> cp /usr/share/doc/rapa-3.0.0/devel.tar.bz2 .
$> tar xvf devel.tar.bz2

If you update rPath Appliance Platform Agent on the local system, you must copy and unpack devel.tar.bz2 again to ensure the files in your development environment correspond to the installed appliance agent version on the system.

Verify you have the following files in the development directory:

• dev.cfg: The configuration file for the development environment; the server.environment value in this file is set to development
• plugingenerator.py: A plugin generator
• plugintemplate/: Contains the templates used by the plugingenerator.py script.
  • plugintemplate/index.kid: Kid template for appearance of your plugin
  • plugintemplate/web.py: The web visible side of the plugin
  • plugintemplate/srv.py: The backend service side of the plugin
• raa: A symbolic link to the local rPath Appliance Platform Agent installation, such as /usr/lib/python2.4/site-packages/raa
• raaplugins: A symbolic link to the local Appliance Agent plugins /usr/lib/raa/raaplugins
• raa-service: The executable used by the startup script for the raa service daemon
• raa_service_dev.conf: The configuration file for the raa service daemon; includes the pluginDirs line that instructs the appliance agent to look for plugins under the standard plugin directory and the local directory
• raa-web-devel: A startup script for the agent's web layer
• README.devel: Contains help information on what each file contained in the archive does
• start-raa-service.sh: A startup script for the raa service daemon

After unpacking the files, make the following modifications to configure the development environment:

1. Use a text editor to create a new file raa_service.conf, and add the following line to that file, using your user name in place of <current_user>:
   listenerUser     <current_user>
   This provides authentication for the UNIX socket through which the service communicates with the web component, verifying that you are running the web component as the current user instead of the default raa-web user.

After this configuration, run the raa-web-devel and start-raa-service.sh scripts to launch raa in development mode:

$> ./raa-web-devel
$> ./start-raa-service.sh

To access the rPath Appliance Platform Agent web interface in development mode, use localhost:8080 as mentioned in the output messages of the raa-web-devel script.

Between steps to change the custom plugin...

Stop and restart the raa-web-devel script to reread the plugins, and refresh the web interface to see the changes.

Between development sessions...

Be sure to stop the raa service each time before starting the raa-web-devel and start-raa-service.sh scripts again for development.

1. Create a file called custom.cfg in the current direcotry that has the following contents (replace username with your Linux user name):

   [global]

   server.run_user = 'username'

   server.run_group = 'username'

   server.lock_path = '.'

2. Download the Cherrypy 3.0.3 tarball (from http://download.cherrypy.org/CherryPy/3.0.3/CherryPy-3.0.3.tar.gz) to your system, and extract it somewhere it will be saved.
3. Create (or append to) the environment variable PYTHONPATH to point at <where you extracted>/Cherrypy-3.0.2. For example, if you extracted the tarball to a directory /home/username/cherrypy, you would run:

   export PYTHONPATH=/home/username/cherrypy/CherryPy-3.0.3

4. create test HTTPS certificates in raa-devel

   ../raa/distro/bin/raa-gendummycert.sh $(pwd)/test.key $(pwd)/test.crt

Starting rAPA 3

rAPA listens on port 8003 for https and 8004 for http by default, which is the same as the system Appliance Agent. Make sure that rAPA is stopped on your system before launching it. You can now start rAPA by running raa-web-devel nd start-raa-service from 2 different terminals (requires sudo installed and configured for the current user). If you open a new terminal, you will need to set your $PYTHONPATH again in that terminal (step 8).

Note:You may need to install ConfigObj and PyPAM on some systems as they are required.

$> sudo service raa stop
$> ./raa-web-devel

$> ./start-raa-service.sh

The url needed for rAPA login is http://localhost.localdomain:8004/ or https://127.0.0.1:8003/

Example Plugin

The Hello World example plugin is available here:
http://people.rpath.com/~jtate/hello-world.tar.bz2

Appliance developers who are developing plugins for rPath Appliance Platform can use the following examples as a guide to the plugin structure and a starting point for the development process.

"Hello world" example plugin

Download and unpack the files in the development environment.

Services plugin

Navigate to /usr/lib/raa/raaplugins/services, where the plugin is installed, to view the file structure of the Services plugin.

Incorporate at least one of these plugins to ensure the plugin development environment works correctly with at least one plugin that will not change. Use the following steps to add the "Hello World" plugin as an example:

1. Download the Hello World example plugin.
2. Unpack the plugin in your plugin development directory.
3. Verify the directory myplugins is created that includes the subdirectory hello.
4. Stop and restart the raa-web and start-raa-service scripts using in the development environment to reread the list of available plugins.
5. Browse to localhost:8080.
6. Verify the Hello, world! link is displayed on the left, and click the link to verify the plugin displays the header "Hello, world!"

Plugin Layout

In a plugin development environment used for rPath Appliance Platform plugin development, use this page as a reference to lay out the plugin as it should appear in the rPath Appliance Platform web interface.

Initial Layout

Copy and modify files from example plugins or other plugins as a basic file structure that will support the new plugin layout. In the "Hello World" plugin example, this would correspond to copying the following files to the corresponding places in the project and plugin directory for your custom plugin:

• <development_directory>/myplugins/hello/index.kid
• <development_directory>/myplugins/hello/web/hello.py

For custom plugin exampleconfig for project example, the copies are:

• <development_directory>/example/exampleconfig/index.kid
• <development_directory>/example/web/exampleconfig.py ("hello.py" changed to "exampleconfig.py" to match the plugin)

Open the web/<custom_plugin>.py file and modify the appropriate items using the comments in the file as a guide. The key points of modification are:

• Python class name ("Hello" in the example)
• Display name ("Hello, world!" in the example)
• Tool tip name ("Hello, world!" in the example)
• Kid template rendered by raa.web.expose decorator ("myplugins.hello.index" in the example)

Open the index.kid file and modify the appropriate items displayed in the web interface. Modify the following items to match the title of the custom plugin:

• Page title (show in the title tags)
• Header text (show in the h1 tags)

Add text or other placeholders if desired to lay out the page appearance prior to adding the plugin functions.

Restart the raa-web script and refresh localhost:8080 to view the plugin. Make adjustments to index.kid (a kid template) and refresh the interface until the items are laid out as desired on the screen.

Final Layout

After you have stepped through developing the plugin functions, incorporate the functions in the plugin layout.

Plugin Development Process

Use the process outlined on this page as part of Plugin Development to develop an rPath Appliance Platform Agent plugin for an appliance project.

Prior to coding your first plugin, rPath recommends performing the following tasks:

1. Set up the plugin development environment.
2. Deploy an example plugin for reference.
3. Create the initial visual layout of the plugin in the rAP web interface.

Outline Actions

Before coding, outline the actions accomplished by the plugin. Typically, a plugin is designed for a single action. However, a partitioned or tabbed interface could be established using the same plugin for multiple related actions. For each action, identify:

• Conditions in which a user should be able to perform the action (such as only for the admin role or only if accessing locally)
• Input required from the rPath Appliance Platform user
• System operations that should occur to process the input and produce the desired results
• Output that should be displayed to the Appliance Platform user

Code the Action

The plugin action is driven by plugin tasks handled by the web and service components, and the web component can request either immediate or scheduled execution by the service component. How these tasks are coded has no specific instructions, but rPath provides the following references and examples that can be used when creating the Python code related pieces used to accomplish the task:

• Use plugin decorators in the plugin code to modify plugin behavior in ways that the Appliance Platform can interpret and use.
• Create plugin database tables if desired to handle data beyond simple input and output operations.
• Add notification functionality to notify rAP users by email regarding an important event related to the plugin.

Code the Input Mechanism

REMAINING CONTENT IN DEVELOPMENT

Code the Output Mechanism

REMAINING CONTENT IN DEVELOPMENT

Plugin Tasks

Types of Tasks

The appliance agent provides a scheduling model for tasks that features three types of task scheduling:

• Immediate Tasks scheduled to execute once, usually within 5 seconds; this is equivalent to manually launching the task
• One Time Tasks scheduled to be executed once; these often include long-running tasks to be scheduled as resources permit
• Recurring Tasks with flexible and repeatable schedules, such as hourly, daily, weekly, and monthly

As described in rPath Appliance Platform Plugin Structure, the action of an Appliance Platform plugin is driven by tasks handled by the web and service components, and the web component can request either immediate or scheduled execution (recurring or one time) by the service component.

Immediate Tasks

Use immediate tasks for actions which complete quickly and require elevated privileges (such as changing the root password). The call for execution made by the web component waits for the service component to complete the task. This should only be used for operations that will take less than 4-5 seconds to complete. An immediate tasks is equal to scheduling the task to occur immediately.

One Time Tasks

Use one time tasks for actions which need to be executed a single time. A plugin that utilizes this type of scheduling is the backup plugin. The task executed by this plugin runs a single time when backup now is selected.

Recurring Tasks

Use recurring tasks for actions which need to be scheduled. The schedule is adjustable to any period of time: weekly, monthly, daily, and so on. A plugin that utilizes this type of scheduling is the backup plugin when you select schedule updates.

Implement a Task

To implement any of these tasks, create an instance from one of the available schedule classes that derive from raa.db.schedule.Schedule. Some of the available schedule classes repeat periodically.

When rAPA executes the service component of the plugin, the schedule ID and a unique execution identifier (execution ID) are passed to the service component. A repeating schedule can cause the service component to be executed multiple times. Each execution of the component corresponds to the same schedule ID and has its own execution ID, which is unique across the system.

The service component may require additional knowledge about the task it is executing. If so, it can perform an XMLRPC call to its associated web component to determine such information. The service component can refer to the proxy for its web component as self.server, and can refer to the root controller proxy as self.rootserver.

Use self.schedule() with the previously created schedule as an argument to initialize the scheduled task. This method returns a schedule identifier (schedule ID). If you need to remember information about this specific tasks for use in other tasks, store information associated with the schedule ID in the plugin's database table.

Immediate Tasks

Scheduled Tasks (One Time and Recurring)

Reusable Code

rPath Appliance Platform Agent plugins can reuse code provided by rAPA and can call functions from other rAPA plugins. Use the following sections as a guide when reusing code and functions.

Reusable Code in Python Classes

The following Python classes in the appliance agent can be used as needed to add a feature to the web interface for a plugin. Read the Python code in each corresponding .py file to see how arguments are processed:

Reusable Code in Kid Templates

To use the reusable code in a plugin, write the kid template for the plugin to import the Python classes, and call methods from those classes where appropriate:

• Use an import line for each class in the Python code section:
  from raa.widgets.<module_name> import <class_name>
• Call the Python methods within the kid template where appropriate:
  ${<method_call>}

Examples of reusable code are in the UpdateTroves plugin file raaplugins/updatetroves/trovelist.kid. The following two lines show the importing of the tabbed page and callback display classes:

from raa.widgets.tabbedpage import TabbedPageWidget
from raa.widgets.callbackdisplay import CallbackDisplayWidget

These lines show calls to methods in those classes:

${TabbedPageWidget(value=pageList)}
${CallbackDisplayWidget(schedId=schedId, optype=optype, statusmsg=statusmsg, status=status)}

Reusing Functions Across Plugins

rAPA plugins can make use of the functions in other plugins. Server sides of plugins can only call server-side functions, and web sides of plugins can only call web-side functions. Use the following syntax from the class definition of one plugin to call the Python method as provided in another plugin:

self.plugins['/<plugindir>/<PluginClass>'].<method>(<variables>).

Replace the <plugindir> and <PluginClass> as appropriate to match the URL that corresponds to the plugin, and replace <method> and <variables> to complete the method call as defined in that plugin. This code goes through rAPA's core code to identify the other plugin and to call the method. This is applicable both for the standard rAPA plugins as well as any custom plugins.

For example, suppose the web side of your custom plugin needs to call the function that gets information about the last update check performed from the Updates plugin. First, note the following:

• The plugin resides in the updatetroves subdirectory of raaplugins and is accessible with the URL https//<hostname>/updatetroves/UpdateTroves.
• The UpdateTroves class on the web side of the plugin includes getLastCheck which does not require any variables to be passed.

With this information, add the following to the web side of your custom plugin to perform this same function:

self.plugins['/updatetroves/UpdateTroves'].getLastCheck().

Plugin Decorators

The rPath Appliance Platform Agent provides decorators for the Python plugin code to modify the behavior of the plugin. The agent provides two decorators in its framework: raa.web.expose (required) and raa.web.require (optional). If it is used, raa.web.require should be on the line after raa.web.expose.

raa.web.expose

The raa.web.expose decorator tells the rPath Appliance Platform how to expose the method: with a template, with XML-RPC, or with JSON. Use arguments together as needed:

raa.web.require

The raa.web.require decorator provides control access to a method. This could be used to allow logged-in user, anonymous, or localhost-only access to the method. This optional decorator is used in conjunction with a raa.web.expose. In the absence of raa.web.require, the default is to be logged in as a user in the role of admin (@raa.web.require(raa.authorization.AnyPermissionPresent('admin'))).

Plugin Database Tables

Developers can create and define database tables as necessary for an rPath Appliance Platform plugin.

Each plugin inherits methods (self.setPropertyValue, self.getPropertyValue, self.getPropertyDict) from the rAAWebPlugin base class. If the plugin only needs a way to set and retrieve key-value pairs, additional database tables may not be necessary.

If the plugin requires a different data storage system than is provided by the property methods, these tables should follow the naming convention plugin_project_tablename where project is replaced by a string identifying the plugin author or organization, and tablename is replaced by a unique identifier for the particular table. For example, the table used to store data used in configuring services is named plugin_rpath_Services. For consistency, use the plugin name for the tablename portion of the table name.

To create a table, define a class in the Python code that inherits from raa.db.database.DatabaseTable or one of its subclasses such as raa.db.database.KeyedTable or raa.db.data.GenericDataTable.

Either your database class or one of your super-classes must have a createSQL field that defines how the table is created. One of the super-classes will likely contain all the basic functions for inserting values into and selecting values from the table, though you may use your own table class to create convenient methods specific to the purpose of the table.

Reference a table from the web component of your plugin by creating a field named tableClass. The value of the field can be one of two things:

• The table class itself with your plugin referring to an instance of the table using the expression self.table
• The value of tableClass as a dictionary whose keys are strings and whose values are table classes, with instances of these tables can referenced in the plugin by accessing the attribute with the same name as the corresponding key in the dictionary.

Plugin Static Files

Plugin developers can include static files such as images, cascading style sheets, and JavaScript code for use by an rPath Appliance Platform plugin.

Add these files to the directory <project_name>/<plugin_name>/web/static. Then, refer to each file in kid templates with a variable in the following format:

${tg.url('/<plugin_name>/static/<file>')}

In this example, <plugin_name> corresponds to the string used to name the plugin directory, and <file> is the static content's filename. Any subdirectory of the static directory can also be added to the path. In the following example, a file named button.png is organized under a subdirectory named images:

${tg.url('/<plugin_name>/static/images/button1.png')}

User Roles

Each rPath Appliance Platform Agent (rAPA) plugin includes one or more roles used by the UserManagement plugin. A role provides access to a plugin or plugin functions, and groups created in User Management are assigned one or more of these roles.

Currently all standard rAPA plugins include the admin role, and users must be in a group with the admin role to see and use the plugin. The standard plugins do not have any function-specific roles defined.

[[Template:note=The admin role is added automatically whether or not there are any additional roles defined. No additional decorators are required for admin, but the kid templates should ensure that admin users have all functionality.]]

To designate roles that should have access to all plugin functions, set a value for the roles variable inside the plugin class. The values listed are available for selection when creating a group in the User Management plugin. The following line adds three roles to the plugin:

    roles=['view', 'execute', 'configure']

If there are no additional changes to the plugin class, the plugin is only displayed to users in groups with one of the displayed roles, and all plugin functions are available to those users.

To define specific functions for a role within a plugin:

1. Use a decorator before the index definition for all roles.
2. Use a decorator and definition in the plugin class for each role.
3. Add the appropriate code to the front end files of the plugin.

The decorator before the index definition includes all the roles that should have access to the plugin and for which the plugin has defined specific functions:

@raa.identity(raa.authorization.AnyPermissionPresent("view", "configure", "execute"))

Roles are statically defined by a plugin, and the roles variable has undefined behavior if it is changed at runtime by a different operation.

Packaging a Plugin

To make an rPath Appliance Platform plugin available for appliance development after development, package the plugin for Conary to install the plugin directories to /usr/lib/raa. By default, the rPath Appliance Platform checks this directory at startup for plugins.

The directory structure should resemble the following:

• /usr/lib/raa
  • <project_name>
    • <plugin_name>
      • srv
      • web
        • static

These should reflect the directories created in the development environment. The paths to the srv and web directories for SamplePlugin in project example would be:

/usr/lib/raa/example/SamplePlugin/srv
/usr/lib/raa/example/SamplePlugin/web

The <project_name> convention is used only as the top-level namespace for organizing plugins originating from the same source.

Test before packaging

Copy the plugin directory to /usr/lib/raa on the local system or a test system to test functions prior to packaging.

Test after packaging

After packaging, install the package on the local system or test system and test its functions again.

There must be a custom configuration that enables the plugin for the appliance, and this is typically packaged and controlled separately from the plugin. See rPath Appliance Platform Agent:Configuration for more information.

ADD SAMPLE PACKAGE RECIPE HERE

Troubleshooting the Package

Content needed...