Version 34 (modified by 13 years ago) (diff) | ,
---|
Contents
Orchestrates AWS cloud resources using boto and pychef
Notice: This plugin is unmaintained and available for adoption.
This fairly new plugin is now being put through its paces at juju.com. I welcome others to try it out and submit tickets for bugs and new features. - robguttman, 7 Feb 2011
UPDATE: The latest version (0.2) now launches and bootstraps new instances in a separate, spawned process which survives Apache timeouts and restarts. See below for more details and screenshot. - robguttman, 13 Mar 2011
Description
This plugin is meant to fill the coordination gap between AWS and Chef, especially:
- Launch an ec2 instance, bootstrap it with chef and apply one or more chef roles
- Terminate ec2 instances and delete its chef node (and client)
There are additional features and quite possibly more to come, but my intention is to keep the scope down to what's needed to orchestrate between AWS and Chef (and not re-implement either the AWS Management Console or the Chef Server webui).
This plugin is highly configurable but is opinionated in some ways. Some of these opinions could be relaxed (submit a ticket!) but they currently are:
- Uses knife bootstrap to bootstrap instances - and so your instances' OS must support this method.
- Assumes the same credentials and security (e.g., keypair) are used for all instances.
- Allows one or more roles (but no recipes) to define a node's run_list.
This plugin relies heavily upon boto and pychef. Special thanks to coderanger for his work and responsiveness on pychef.
Configuration
- Install the plugin (after downloading and unzipping):
cd cloudplugin/0.12 sudo python setup.py bdist_egg sudo cp dist/TracCloud*.egg /your/trac/location/plugins/
See TracPlugins for more installation details and options. You'll likely need to restart Trac's web server after installation.
- Enable the plugin in
trac.ini
:[components] cloud.* = enabled
You can alternatively use the Trac Web Admin GUI to enable any or all rules.
- Configure the main
trac.ini
section. A minimal configuration would look like this:[cloud] aws_key = <aws-access-key> aws_secret = <aws-secret-key> aws_keypair = <ec2-keypair-name> aws_keypair_pem = /path/to/keypair.pem aws_username = <ec2-username> chef_base_path = /path/to/.chef
AWS access configuration
The aws_*
options (see example above) are for your AWS credentials and security information. For more info, see here and here.
Chef access configuration
Your Trac server must already have been configured using knife configure
. For more info see here. Set the chef_base_path
option to the standard chef directory that contains the knife.rb
and client pem files needed to connect with the chef server. Common settings for this option are /etc/chef
or /home/<username>/.chef
. Be sure that the validation.pem
file is located where the knife.rb
file says it should be.
Bugs/Feature Requests
Existing bugs and feature requests for CloudPlugin are here.
If you have any issues, create a new ticket.
Download
Download the zipped source from [download:cloudplugin here].
Source
You can check out CloudPlugin from here using Subversion, or browse the source with Trac.
Examples
This plugin is designed to support multiple AWS resource types, the most notable being ec2 instances.
Instances
The configuration for the ec2 instance resource type has reasonable defaults all of which can be overridden in its own trac.ini
section:
[cloud.instance] ...
The different configuration options are broken out below.
Labels and such
[cloud.instance] title = Instances order = 1 label = Instance description = AWS EC2 instances.
The title
and order
options are used for contextual navigation - see screenshot above. The order
should start with 1
. If order
is set to an empty string then the resource will not be shown in the contextual menu (but is still accessible by its url). The label
value is displayed in various buttons and forms for the name of a single resource item. The description
option is displayed in the grid view - see screenshot above - much like a Trac report description.
Field Definitions
The fields shown in each view can be configured. Fields are first defined much like Trac custom fields but prefixed with field.
:
[cloud.instance] .. field.name = text field.name.label = Name field.name.handler = NameHandler field.run_list = multiselect field.run_list.label = Roles field.run_list.databag = roles # special token field.run_list.handler = RunListHandler field.ec2.placement_availability_zone = select field.ec2.placement_availability_zone.label = Availability Zone field.ec2.placement_availability_zone.options = No preference|us-east-1a|us-east-1b|us-east-1c|us-east-1d field.ec2.placement_availability_zone.value = us-east-1c .. many more fields ..
Each field.*
option must map exactly to its chef attribute name. For example, ec2.placement_availability_zone
above maps to the ec2
attribute namespace and the placement_availability_zone
attribute within that namespace in chef. It's defined as a select
field with a display label and several options separated by a pipe (|
) just like Trac custom fields. There are some important differences to Trac custom fields, however:
- A field can define a data bag instead of options
- A field can define a "field handler"
Data Bags for options
You may prefer to define a select (or multiselect) field's options by using a chef data bag instead of listing them in the trac.ini file. For example, a field defined like this:
[cloud.instance] .. field.ec2.ami_id = select field.ec2.ami_id.label = Image ID field.ec2.ami_id.databag = ec2_ami_id
will query chef for and use the ec2_ami_id
data bag items. The individual data bag items should be formatted like this:
{ "id": "ami-xxxxxxxx", "value": "ami-xxxxxxxx", "name": "ami-xxxxxxxx (64-bit)", "order": 1 }
The name
will be used as the option's displayed name and the value
will be used as the option's value. If an order
field is provided, it will be used to order the options accordingly (much like custom field ordering).
Specifying roles
as a data bag has a special meaning (see the run_list
field example above) - chef will be queried for the list of roles instead of a data bag.
Field handlers
A field handler converts a raw chef attribute value into something more human-readable and vice-versa. For example, the RunListHandler
defined above for the run_list
field extracts roles from a chef node's run_list and displays them in an easy-to-read way. It also converts data the other direction - i.e., it converts the list of roles from a web form into the correct format to set the node's run_list
.
Other field handler examples include displaying a url to a web site at a specific port, converting epoch's to dates and times, etc. See handlers.py for the full list of provided field handlers and defaults.py for examples of which fields they're used for by default
Views
In addition to field definitions, you can define which fields should be viewed in which views, their order, and whether or not a field should be read-only in that view. Example:
[cloud.instance] .. crud_view = name, ec2.instance_type, ohai_time, .. crud_new = roles, created_by, created_at, .. crud_edit = roles, created_by, created_at*, .. grid_columns = name, ec2.instance_type, ohai_time, .. grid_sort = ohai_time grid_asc = 0
The crud_view
, crud_new
, crud_edit
, and grid_columns
options are lists of field names in the order to be displayed in their respective views. ("crud" stands for "create, read, update, delete".) An asterisk (*
) appended to a field name (only applicable for crud_new
and crud_edit
) indicates that the field should be read-only in that view. All fields in the crud_new
and crud_edit
views will be used to set or update the chef resource item.
The grid_sort
and grid_asc
are for default sorting in the grid view.
Progress views
Launching or bootstrapping an ec2 instance can take a variably long time - usually longer than the default Apache timeouts (e.g., FcgidIOTimeout
). Instead of wrestling with Apache timeout configurations, this plugin now launches and bootstraps ec2 instances in a separate, spawned process that should survive Apache timeouts and restarts. Surviving Apache restarts is also helpful when Trac itself is chef-managed and may be restarted by the chef-client
upon config changes. Creating a new instance now brings up a new 'Progress' page that tracks the spawned process' progress:
As the instance id and public DNS become known, they're presented.
Recent Changes
- 14004 by rjollos on 2014-07-12 18:27:48
-
Correct
license
keyword argument insetup.py
. Refs #11832.
- 13996 by rjollos on 2014-06-25 00:56:32
-
Fixed author in COPYING file. Refs #11832.
- 13994 by rjollos on 2014-06-25 00:51:36
-
Changed to 3-Clause BSD license with permission of author. Refs #11832.
(more)
Author/Contributors
Author: robguttman
Maintainer: robguttman
Contributors:
Attachments (4)
- nav.png (2.2 KB) - added by 13 years ago.
- instance-new.png (28.5 KB) - added by 13 years ago.
- progress.png (15.6 KB) - added by 13 years ago.
- instance-grid.png (118.5 KB) - added by 13 years ago.
Download all attachments as: .zip