How to create base docker images for CentOS

Background

Docker is a lightweight container technology that is gaining a lot of momentum in the development community. Compared with traditional methods of virtualization, Docker allows rapid prototying, building and shipping an application.

I used Docker to simulate the production environment on my laptop. First step in this process is to have an base image that replicates the virtual or real machines used in production. In this blog, I describe how to create base Docker images from Kickstart files which can be used for creating Openstack/AWS virtual machines or Docker container.

Pre-requisite

1. Start a CentOS virtual machine (on VirtualBox, VMWare Fusion or Vagrant). You can download CentOS6.5 virtual machines from [https://github.com/opscode/bento]
2. Login to the machine.
3. Add the EPEL repository from where we shall install the tools required to create images.

   sudo yum install -y http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
   

Creating base image

1. Install image creation tools on Centos VM.

   sudo yum install -y appliance-tools libguestfs-tools
   

2. Create an image in QCOW2 format appliance-creator can be used to automatically install a virtual machine using the kickstart file.

   sudo appliance-creator -c container-65.ks -d -v -t /tmp \
   -o /tmp/myimage --name "centos65" --release "6.5" \
   --format=qcow2;
   

3. Export from QCOW2 to tar

   sudo virt-tar-out -a /tmp/myimage/centos65/centos65-sda.qcow2 / - | gzip --best > centos65.tar.gz
   

4. Import image to docker registry

   cat centos65.tar.gz | sudo docker import - imtiaz/centos65:latest
   

5. Optional step: Publish image to docker hub (public image repository for docker images)

   sudo docker push imtiaz/centos65

How to setup stand-alone Openstack Heat service

Background

Heat is an orchestration service from Openstack. While this service can be installed on an Openstack controller along with other services, Heat can run as an independent service and can be used to provision any remote clouds. This is useful, for example  if you want to deploy a cloud application based HOT or AWS CloudFormation template on an Openstack cloud. 

Setting up Heat service (on Mac laptop)

Pre-requisites

• MySQL (5.5)
• RabbitMQ (or any other message queue)
• Python-mysql module

NOTE According to the documentation, it should be possible to setup Heat without having to install MySQL or RabbitMQ but I ran into a number of issues in trying to do that. 

Install MySQL

Download MySQL 5.5 64 bit and install on your Mac. I used the DMG package that is available. Please note, newer versions of MySQL didn't work with Heat and required a lot of configuration to get it to work. 

Once installed, you can start MySQL service from "Systems Preferences" Window. 

Create Heat DB on MySQL

After installing MySQL, create a heat DB. 

> mysql -u root CREATE DATABASE heat CHARACTER SET utf8 ; GRANT ALL PRIVILEGES ON heat.* TO 'heat'@'localhost' IDENTIFIED BY 'heat_dbpass'; GRANT ALL PRIVILEGES ON heat.* TO 'heat'@'%' IDENTIFIED BY 'heat_dbpass';

Install Message Queue

Download RabbitMQ and install. 

Start the RabbitMQ service. 

Install Python MySQL 

Installing MySQL client on Mac proved quite challenging as it required some compilation of some libraries and Mac's gcc compiler didn't understand the compiler flags. Here are are steps to overcome these issues: 

1. Install XCode 4.1 (or latest) from App store. 
2. Install MySQL python client using the following command. 

export DYLD_LIBRARY_PATH=/usr/local/mysql/lib export PATH=/usr/local/mysql/bin:$PATH sudo ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install mysql-python

Installing Heat Service

1. Check out the source from github.
   

   git clone https://github.com/openstack/heat.git

   
   
2. (Optional) Install Heat by running sudo python setup.py install from the heat directory
   
   

Configuring Heat

In order to run Heat services, it must be registered with Keystone. This Keystone service can run locally (on Mac) or anywhere else (a Vagrant box for example). The following steps show how: 

export SERVICES_TENANT_NAME="service" # or 'services'  #set HEAT_HOSTNAME to IP address of the server running Keystone. export HEAT_HOSTNAME=   $ keystone user-create --name heat --pass ${HEAT_USER_PASSWORD_OF_CHOICE} --tenant ${SERVICES_TENANT_NAME} $ keystone user-role-add --user heat --role admin --tenant ${SERVICES_TENANT_NAME} $ keystone service-create --name heat --type orchestration   # Note the service Id after this command. This is the HEAT_SERVICE_ID used in the commands to follow.  $ keystone service-create --name heat-cfn --type cloudformation $ keystone endpoint-create --region RegionOne --service-id ${HEAT_CFN_SERVICE_ID} --publicurl "http://${HEAT_HOSTNAME}:8000/v1" --adminurl "http://${HEAT_HOSTNAME}:8000/v1" --internalurl "http://${HEAT_CFN_HOSTNAME}:8000/v1" $ keystone endpoint-create --region RegionOne --service-id ${HEAT_SERVICE_ID} --publicurl "http://${HEAT_HOSTNAME}:8004/v1/%(tenant_id)s" --adminurl "http://localhost:8004/v1/%(tenant_id)s" --internalurl "http://localhost:8004/v1/%(tenant_id)s" # where HEAT_SERVICE_ID is the ID of the Heat service

Setting Heat.conf file

In the etc/heat/heat.conf file, set the properties for database, RabbitMQ and Keystone endpoints. The following example shows the minimum required configuration to start Heat services.  

[DEFAULT] # plugin_dirs=/Users/imtiaz.chowdhury/git/heat/contrib   sqlite_db=heat.sqlite sqlite_synchronous=false   rabbit_host=localhost rabbit_port=5672 rabbit_userid=guest rabbit_password=guest rabbit_virtual_host=/     [auth_password]   [database] sql_connection=mysql://heat:heat_dbpass@localhost/heat   [keystone_authtoken] auth_host=localhost auth_port=35357 auth_protocol=http auth_uri=http://localhost:35357/v2.0 admin_user=heat admin_password=heat_password admin_tenant_name=services

Starting Heat

1. Start Heat API service. This will start the REST API services
   

   heat-api --d --config-dir <directory of the Heat configuration, e.g. /Users/imtiaz/git/heat/etc/heat >

2. Start Heat engine.
   

   heat-engine --d --config-dir <directory of the Heat configuration, e.g. /Users/imtiaz/git/heat/etc/heat >

Verifying Heat installation

Now that Heat is installed, you should be able run Heat client and get stack information. First, you'll need to install the Heat client. 

sudo pip install python-heatclient

Next set the username, password and Keystone endpoint the Heat client will use to authenticate before running any Heat commands. For example,

OS_PASSWORD=b6211c965173421b OS_AUTH_URL=http://localhost:5000/v2.0/ OS_USERNAME=admin OS_TENANT_NAME=admin

NOTE The above setting will allow Heat to create resources for admin project as admin user. If you want to use Heat to create resources for other tenants then you should use user credentials associated with that tenant 

Running heat stack-list now return an empty list. 

You should now be able to launch VM instances or create other resources by using a Heat template. Many sample templates are available on https://github.com/openstack/heat-templates.