In this article, we provide an overview of the OpenStack open source cloud computing environment. It is a modular design that divides user authentication and provisioning, storage, and networking into distinct modules, also known as projects or nodes. We describe each of these and give examples of how the administrator can interact with them using command-line tools, the dashboard, and APIs. We explain that, in order to use OpenStack as it is designed (to provision and deploy a large number of virtual machines), it is best to take the time to learn how to write Python code to script all of that. We introduce the reader to some OpenStack concepts that the reader might not already know, such as object storage.

OpenStack has lots of powerful corporate backers and is rapidly moving ahead of competing products and gaining on the market leaders. The market leaders have recognized this, so they have worked with the open source project to make sure their product works with OpenStack. Any administrator who works with cloud computing systems should take a look at OpenStack even if only to enhance their understanding of the product they currently use.

Introducing OpenStack

OpenStack is becoming the standard open source cloud hosting software. Lots of companies are using it. It is designed to deploy one or two virtual machine or hundreds or thousands of them. OpenStack installs the major hypervisor, and in addition it can install the operating system or hypervisor directly onto a machine that has no operating system at all. This is called bare-metal.

The new release of OpenStack, called Kilo, has just shipped. The OpenStack open source project uses letters for its releases, so ‘K’ as in ‘Kilo’ is the 11th release since Nasa and Rackspace began the OpenStack project in 2010.

OpenStack optionally ships with Ubuntu and Redhat, showing how popular it has become. Among its many contributing developers are Microsoft and VMware. Of course those companies who make hypervisors that you have to pay for want to be involved so that the VMware and Hyper-v hypervisors are included among the virtual machines you can deploy with OpenStack, which is free. OpenStack uses KVM as the default virtual machine, which is pre-built into Linux (‘KVM’ means Kernel Virtual Machine). You can also deploy OpenStack to bare metal machines. Below we go into detail about this.

Not a Push Button Graphical System

OpenStack is far more complicated than, say, VMware vSphere, and some other proprietary products. But if you consider that it makes possible deploying many machines at the same time using a script, it is easier than using a GUI. OpenStack has no GUI, or not really. It’s really just a set of APIs with a basic Dashboard giving a visual view into your deployed Infrastructure. You can program OpenStack using Python or use its command-line tools. The OpenStack system itself is written in Python. The OpenStack Python SDK is here.

Below we discuss most of the major features of OpenStack. The best approach to explaining it is to list its features instead of writing dense paragraphs because it is so large. This introductory article should then lead you to other articles and videos for further details and additional study resources.

OpenStack Architecture

The diagram below from OpenStack give an overview of the entire system. Below we explain some of the modules (called ‘projects’ and ‘nodes’).

In sum:

• Heat orchestrates (i.e. schedules and controls) the deployment of virtual machines.
• Cellometer is responsible for billing clients.
• Keystone does authentication and user ID provisioning.
• Swift, Glance, and Cinder manage storage and files.
• Neutron configures subnets and virtual networks and internet gateways for the virtual machines.
• Horizon is a web dashboard.

There are additional nodes too, for example a database node.

Nova: Compute

Each part of the OpenStack system has its own icon. Above is the one for Compute.

You can think of Compute as the piece that provisions virtual machines. In other words, if you are running a cloud and someone requests a virtual machine, you create it using Compute.

Compute creates these virtual machines and containers: Libvirt (KVM, QEMU, Xen, LXC), XenAPI, Hyper-V, VMware ESX, PowerVM, Docker, and bare-metal. Docker and LXD, which is new with Ubuntu 15.04, are containers, meaning they are separate Compute environments but are not full hypervisors.

Working with OpenStack

You interact with OpenStack through its APIs or the command line. The command line for Compute is Nova.

Here is a sample command:

ironic node-create -d driver -c chassis -i key=value -p key=value -e key=value -u uuid -n name

This registers a node with the ironic service. Ironic is part of Computer.

You can also communicate with REST APIs (so it works just like Amazon EC2, which is also a supported format) which creates and uses JSON objects to work with web services to create VMs.

In this example below, taken from OpenStack, we use curl to log in to the Keystone identity service to receive a session token. We send it a JSON object since that is what its API requires.

curl -i \
-H "Content-Type: application/json" \
-d '
{ "auth": {
  "identity": {
   "methods": ["password"],
   "password": {
    "user": {
     "name": "admin",
     "domain": { "id": "default" },
     "password": "adminpwd"
    }
   }
  }
 }
}' \
http://localhost:5000/v3/auth/tokens ; echo

Keystone returns the response also in JSON format. The user login (token) is inside the JSON object.

HTTP/1.1 201 Created
X-Subject-Token: MIIFvgY...
Vary: X-Auth-Token
Content-Type: application/json
Content-Length: 1025
Date: Tue, 10 Jun 2014 20:55:16 GMT

{"token": {"methods": ["password"], "roles": [{"id":
"9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}, {"id":
"c703057be878458588961ce9a0ce686b", "name": "admin"}], "expires_at":
"2014-06-10T2:55:16.806001Z", "project": {"domain": {"id": "default", "name":
"Default"}, "id": "8538a3f13f9541b28c2620eb19065e45", "name": "admin"},
"catalog": [{"endpoints": [{"url": "http://localhost:3537/v2.0", "region":
"RegionOne", "interface": "admin", "id": "29beb2f1567642eb810b042b6719ea88"},
{"url": "http://localhost:5000/v2.0", "region": "RegionOne", "interface":
"internal", "id": "8707e3735d4415c97ae231b4841eb1c"}, {"url":
"http://localhost:5000/v2.0", "region": "RegionOne", "interface": "public",
"id": "ef303187fc8d41668f25199c298396a5"}], "type": "identity", "id":
"bd73972c0e14fb69bae8ff76e112a90", "name": "keystone"}], "extras": {},
"user": {"domain": {"id": "default", "name": "Default"}, "id":
"3ec3164f750146be97f21559ee4d9c51", "name": "admin"}, "issued_at":
"201406-10T20:55:16.806027Z"}}

These messages are transmitted to the servers using messaging software like RabbitMQ.

You can also work with OpenStack using Python or even create JSON objects right in the browser using JSON plugin editors for Firefox or Chrome.

Each of the OpenStack projects are divided into components. For example, Compute includes:

• nova-api service
• nova-cert module
• nova-scheduler service
• and others.

Ironic: Bare Metal Provisioning

Ironic is part of Compute. OpenStack Computer can start a server and install a hypervisor directly on an empty computer. This means that it can make bios settings, such as changing the boot device order. It uses PXE boot and IMPI to work with the machine.

As you know, there are lots of ways to set up a Hypervisor. You can install an operating system and then put a hypervisor on top of that. Or you can do it the other way around. Ironic does all of this.

Supported Hypervisors

Look at this graphic of functions supported for each Hypervisor. (The whole list will not fit on the page.) This list is a good way to understand what OpenStack Compute does. As you can see it can stop and start instances, assign storage, as well as do much more.

Swift: Object Storage

Swift provides object storage for OpenStack. So Swift does what Amazon S3 does. And like Amazon S3, you interact with it using REST APIs. (You can also use Swift without using any other part of OpenStack, in other words just using it to store objects like photos.) Objects stored in object storage cannot be updated; they can only be added and deleted. (An object can be replaced by deleting one and adding another on top of it.) An object could be something like a photo, a virtual machine image, style sheets, or anything that is considered static.

Each file in object storage has an object id. This is different from the traditional hierarchical file storage where files are located by their directory structure:

//mount point/customers/Russia/oil_gas/Exxon/accounts.xls.

In OpenStack, objects are arranged by accounts and containers like this:

http://swift.example.com/v1/account/new_container

Cinder: block storage

Cinder manages block storage. Block storage means creating disks and folders, formatting disks, partitioning disks, and anything else you can do with storage. Disks are divided into blocks with traditional and modern disk storage systems.

Some of the components of Cinder are:

• cinder-api
• cinder-volume
• cinder-backup daemon.

Glance: Image storage

Glance keeps track of virtual machine images for Nova. It also uses REST to query VM images. Images can be stored in traditional hierarchical file systems or object storage. Here is a sample command-line command. This is the command to start the server:

sudo glance-control api start glance-api.conf

Keystone: Identity

The Keystone service supports simple user ID and password authentication all the way up to complicated PKI certificate authentication. It includes a directory like LDAP. When you log in to OpenStack, you send a login token to other services, which you do by sending them via the X-Auth-Token HTTP headers.

Celiometer: Metering

You cannot give your customers virtual machines for free. So Celiometer keeps track of machine usage so you can charge your clients. Of course if you are running your own cloud for your own business you might not actually charge your users.

Neutron: Networking

Neutro, as the name suggests, create networks. It creates subnets, routers, gateways, and assigns IP address. When a customer wants a virtual machine you have to create a network for them, give them an IP address, and create subnets. Neutron does this.

Neutron has plugins for Cisco virtual and physical switches including: NEC OpenFlow products, Open vSwitch, Linux bridging, and VMware NSX.

Here is a sample configuration. As you can see there is an internal network and a public IP address for the internet. Block and object storage nodes are being created in this sample OpenStack environment.

Summary of Features by Module

Each OpenStack module is called a ‘project’ as each is maintained by a team of developers on different open source projects under the main OpenStack project.

Another way to understand OpenStack is by looking at the different items in each project. So we list those items here.

Network Node has:

• Open vSwitch
• Networking ML2 Plug-in
• Networking Open vSwitch Agent
• Networking L3 Agent
• Networking DHCP Agent
• Networking Metadata Agent

Compute Node has:

• KVM Hypervisor
• Open vSwitch
• Compute
• Networking ML2 Plug-in
• Networking Open vSwitch Agent
• Telemetry Agent

Block Storage node has:

• iSCSi Target Service
• Block Storage Volume Service
• Telemetry Agent

Object storage node has:

• Object Storage Account Service
• Object Storage Container Service
• Object Storage Object Service

Controller node has:

• SQL database service
• message queue
• network time service
• identity
• image service
• compute management
• networking management
• networking MLS plug-in
• block storage management
• orchestration
• object storage proxy service
• telemetry management
• telemetry agents
• database service
• data processing service

Horizon: Dashboard

OpenStack includes three dashboards: user, system, and settings. It can be used to provision virtual machines, to add users, and to work with the other OpenStack components.

Developer can extend the dashboard as well to, for example, add custom panels. You can also brand it, adding your company’s logo and screen colors for users. And then there are third party modules you can add in.

Here is a view of the Dashboard:

Conclusion

Wrapping up, at this point you should have a basic understanding of Nova as the main driver of the system and of Heat as orchestrating the deployment of virtual machines. Neutron is what lets you set up internet access and subnets for each of your customers. Horizon gives you a dashboard view into the systems you have built. Glance, Swift, and Cinder keep track of virtual machine images, data stored as objects, and data stored in block storage. Keystone provides a single point of authentication to work with the other nodes. Cellometer provides the data you need for customer billing.

As you can see, OpenStack is quite a large and complicated system. If you want to start small in order to learn it, you need only install Nova, Glance, Keystone, and Neutron to make a working system. Understanding all of OpenStack requires that you understand all aspects of cloud computing: networking, virtualization, networking, object storage, block storage, and even customer billing. Plus, it will require you to pull out the manual and learn something about writing Python code if you haven’t already. On the upside it’s a real career booster as OpenStack engineers are in demand and its market share is growing.