patg.net

home

Docker: Containers for the Masses -- Yet another way to use Ansible to build Docker images

24 Jun 2014

Good Monday morning! Over the weekend, there was a comment to my previous post covering using Ansible to build Docker images from Michael DeHaan, CTO and creator of Ansible (thank you!) reminding me of his work discussed in his blog post Installing and Building Docker with Ansible that is definitely worth sharing and the method I first used to build Docker images and wrote my first role that will be shared in this post.

For the reader just joining, the previous posts in this series “Docker: Containers for the Masses” are:


Building Docker Images with Ansible using a Dockerfile

Michael’s article details how to install Ansible, how to use Paul Durivage’s angstwad.dcoker_ubuntu Ansible role, also features an important-to-know way of building Docker images whereby an image is build using a Dockerfile that specifies the installation of Ansible, checks out your playbook repository which it then runs with Ansible resulting in a built image with everything you would want on that image. This is different than in the previous post that details using Ansible to run the Docker image-building process and is yet another example on how using Ansible and Docker together is flexible and the approach to both interchangeable and each method equally valid depending on what the user requires.

Additionally, I used this methodology when I first started using Ansible and Docker and forked Michael’s repository, adding a Galera role.

The Playbook

In addition to showing yet another way to build Docker images, this post will also give the reader more insight into using Ansible in general and show another example of what one can do with a Dockerfile.

This post will detail a playbook I wrote when I forked the docker_dna repo. In my role with the HP ATG Group, I was tasked with researching Ansible and Docker and wanted to accomplish several things: Learn Docker and Ansible as well as see if my experience – and a [Salt][saltstack] template and methodology for setting up a Galera cluster could be easily ported to Ansible.

Directory layout

The repo, when cloned, there are the base, rabbitmq, zookeeper, and galera subdirectories. The last one was added by myself when I used this repo to get familiar with this methodology for building Docker images. In that subdirectory

$ ls -1
dna.yml
docker-dna_galera.yml
Dockerfile
group_vars
host_vars
README.md
roles


Top-level playbook

The dna.yml playbook sets some variables and includes docker-dna_galera.yml:

---
# file: dna.yml
- include: docker-dna_galera.yml


Tasks

The tasks that then are used for this role which are broken up into specific operations:

$ ls -1 roles/docker-dna_galera/tasks/
clustercheck.yml
configure_galera.yml
grants.yml
install_galera.yml
main.yml
misc.yml
repo.yml


Specifying using the docker-dna_galera role

docker-dna_galera.yml in turn uses the roles common and docker-dna_galera

---
# file: docker-dna_galera.yml
- hosts: docker-dna_galera
  roles:
      - common
      - docker-dna_galera


Role variables

By using the docker-dna_galera role, the role’s variables can be set in the file group_vars/docker-dna_galera which contains variables used by the the templates roles/docker-dna_galera/templates/etc/mysql/my.cnf.j2 and roles/docker-dna_galera/templates/usr/bin/clustercheck.j2, as well as some of the role’s tasks. These values should be changed to whatever you need them to be.

---
# variables for the docker-dna_galera group, which in
# this case is only localhost

clustercheck_username: clustercheck
clustercheck_password: xyz123
xtrabackup_username: xtrabackup
xtrabackup_password: abc987
docker_host: localhost
docker_username: docker
docker_password: c-kr1t

Top-level playbook including tasks

main.yml includes each task in the order it needs to be run:

--
# file: roles/docker-dna_percona/tasks/main.yml

- include: misc.yml
- include: repo.yml
- include: install_galera.yml
- include: grants.yml
- include: configure_galera.yml
- include: clustercheck.yml


Misc playbook

The first task misc.yml installs vim or any other package other than the Percona packages:

---
# file: roles/docker-dna_percona/tasks/misc.yml

- name: Install things I like
  apt: pkg={{ item }}
       state=present
  with_items:
      - vim


Set the repo

The repo.yml task simply sets up apt to use the Percona apt repo:

---
# file: roles/docker-dna_galera/tasks/repo.yml

- name: Obtain Percona public key
  # apt_key: url=http://keys.gnupg.net/pks/lookup?op=get&search=0x1C4CBDCDCD2EFD2A
  apt_key: url=http://www.percona.com/downloads/RPM-GPG-KEY-percona
           state=present

- name: Add Percona repository
  apt_repository: repo='deb http://repo.percona.com/apt precise main'
                  state=present

- name: Add Percona source repository
  apt_repository: repo='deb-src http://repo.percona.com/apt precise main'
                  state=present

- name: Update apt cache
  apt: update_cache=yes


Install the database software

The install_galera.yml task installs Percona XtraDB Cluster as well as copying a startup script into /usr/local/bin. This is somewhat historic as upstart didn’t work with older versions of Docker

---

- name: Install Percona XtraDB Cluster server
  apt: pkg={{ item }}
       state=present
  with_items:
      - percona-xtradb-cluster-server-5.6
      - python-mysqldb
      - xinetd
      - telnet

- name: Copy the helper script
  copy: src=usr/local/bin/mysql_run.sh
        dest=/usr/local/bin/mysql_run.sh
        mode=0755


Set the database grants

The grants.yml task sets the grants for the database that are needed to run a successful Galera cluster

---
# file: roles/docker-dna_percona/tasks/grants.yml

- name: Add Docker database user
  mysql_user: user={{ docker_username }} host={{ docker_host }}  password={{ docker_password }} priv=*.*:"all privileges"

- name: Add xtrabackup database user (for Galera SST)
  mysql_user: user={{ xtrabackup_username }} host="localhost" password={{ xtrabackup_password }} priv=*.*:"grant, reload, replication client"

- name: Add clustercheck database user (for clustercheck/xinetd -> haproxy)
  mysql_user: user={{ clustercheck_username }} host="localhost" password={{ clustercheck_password }} priv=*.*:"grant, reload, replication client"


Configure the database

configure_galera.yml generates /etc/mysql/my.cnf and shuts down the mysqld process. Why shut it down? Because the container this is running on is only for building the image and just as when creating a snapshot, it makes more sense to not have a running database with open file-handles that an image is created from.

---
# file: roles/docker-dna_percona/tasks/configure_galera.yml

- name: Configure Percona XtraDB Cluster server
  template: src=etc/mysql/my.cnf.j2
            dest=/etc/mysql/my.cnf

- name: Stop MySQL
  action: service name=mysql state=stopped


Set up the clustercheck script for HAProxy

The last task, clustercheck.yml, sets up the python script used by HAProxy to determine which master to use. Why not the original xinetd-based clustercheck script? The author was never able to get the xinetd-based clustercheck script working with Docker.

# file: roles/docker-dna_percona/tasks/clustercheck.yml

- name: Copy clustercheck script
  copy: src=usr/local/bin/pyclustercheck
        dest=/usr/local/bin/pyclustercheck
        owner=root
        group=root
        mode=0700


Template generation

The templates for the docker-dna_percona role are the my.cnf.j2 jinja template which is generated as /etc/mysql/my.cnf and transliterates the variables set in the previously-mentioned variables file. This snippet shows the Galera-specific mysql options. The cluster address is set to bootstrap. Remember that this is an image that is being built. One would need to use Ansible to configure this value to reflect node membership state of the cluster when the containers are run that use this image as well as set different passwords.

wsrep_provider          = /usr/lib/libgalera_smm.so
wsrep_slave_threads     = 4
wsrep_sst_method        = xtrabackup
wsrep_sst_auth          = :
wsrep_cluster_name      = percona-cluster
wsrep_cluster_address   = gcomm://
wsrep_provider_options  = gcache.size=2G;


Dockerfile goodness

Finally, the Dockerfile! This is where all the work happens.

# docker-dna/galera/Dockerfile
#
# VERSION    0.1.0
#
FROM capttofu/docker-dna_base
MAINTAINER Patrick aka CaptTofu Galbraith , patg@patg.net

# Update distribution
RUN apt-get update \
      && apt-get upgrade -y \
      && apt-get clean

# Add files
ADD . ./DockerDNA

# Install Percona XtraDB Cluster 
RUN ( echo '[docker-dna_galera]' && \
      echo 'localhost' \
    ) > /etc/ansible/hosts \
      && ansible-playbook ./DockerDNA/dna.yml --connection=local \
      && apt-get clean

# Expose MySQL/Galera
EXPOSE 3306 4444 4567 4568 9200

ENTRYPOINT ["/usr/local/bin/mysql_run.sh"]

The above Dockerfile specifies using the capttofu/docker-dna_base image as a base. This image already has ansible and it’s prerequisite libraries pre-installed and ready to use. The first event that is run in the Dockerfile is to update the apt system. Next, everything in the current repository is copied to a DockerDNA directory in the root directory of the temporary container.

Building the image

Next, by running docker build . in the same directory, the image will be built, using the pre-installed ansible, run with a l;ocal connection, in this case.

docker-dna-galera/galera$ docker build .
Uploading context 49.15 kB
Uploading context
Step 0 : FROM capttofu/docker-dna_base
Pulling repository capttofu/docker-dna_base
1e47da3640f1: Download complete
80cd3d2446e3: Download complete
< ... snip -- numerous lines like the above and below ... >
5607ff993e85: Download complete
27e469823903: Download complete
 ---> 167dc428d943
Step 1 : MAINTAINER Patrick aka CaptTofu Galbraith , patg@patg.net
 ---> Running in ceb1ec12aab7
 ---> c7916cff77cd
Removing intermediate container ceb1ec12aab7
Step 2 : RUN apt-get update      && apt-get upgrade -y      && apt-get clean
 ---> Running in d5b1189506ec
Get:1 http://security.ubuntu.com precise-security Release.gpg [198 B]
Get:2 http://ppa.launchpad.net precise Release.gpg [316 B]
< ... snip ...> 
Hit http://archive.ubuntu.com precise/main Translation-en
Hit http://archive.ubuntu.com precise/universe Translation-en
Get:20 http://archive.ubuntu.com precise-updates/main Translation-en [431 kB]
Get:21 http://archive.ubuntu.com precise-updates/universe Translation-en [180 kB]
Fetched 4734 kB in 20s (228 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following packages have been kept back:
  ansible initscripts upstart
The following packages will be upgraded:
  apt apt-utils base-files ca-certificates curl dpkg file gnupg gpgv ifupdown
  initramfs-tools initramfs-tools-bin iproute libapt-inst1.4 libapt-pkg4.12
  libc-bin libc6 libcurl3 libcurl3-gnutls libdrm-intel1 libdrm-nouveau1a
  libdrm-radeon1 libdrm2 libgnutls26 libmagic1 libssl1.0.0 libudev0
  libyaml-0-2 multiarch-support openssh-client openssl perl-base procps
  python-apt python-apt-common python-software-properties python2.7
  python2.7-minimal tzdata udev
40 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
Need to get 23.0 MB of archives.
After this operation, 14.3 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ precise-updates/main base-files amd64 6.5ubuntu6.7 [61.0 kB]
Get:2 http://archive.ubuntu.com/ubuntu/ precise-updates/main dpkg amd64 1.16.1.2ubuntu7.5 [1829 kB]
< ... snip ... >
Get:40 http://archive.ubuntu.com/ubuntu/ precise-updates/main python-software-properties all 0.82.7.7 [23.5 kB]
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
Installing new version of config file /etc/issue.net ...
< ... snip - lots of apt output ...>
ldconfig deferred processing now taking place
Processing triggers for initramfs-tools ...
 ---> 0a8159128717
Removing intermediate container d5b1189506ec
Step 3 : ADD . ./DockerDNA
 ---> 93766d0fc5b2
Removing intermediate container f4b216a8ddbf
Step 4 : RUN ( echo '[docker-dna_galera]' &&      echo 'localhost'    ) > /etc/ansible/hosts      && ansible-playbook ./DockerDNA/dna.yml --connection=local      && apt-get clean
 ---> Running in e41cd0dba011

PLAY [docker-dna_galera] ******************************************************

GATHERING FACTS ***************************************************************
ok: [localhost]

TASK: [Install things I like] *************************************************
changed: [localhost] => (item=vim)

TASK: [Obtain Percona public key] *********************************************
changed: [localhost]

TASK: [Add Percona repository] ************************************************
changed: [localhost]

TASK: [Add Percona source repository] *****************************************
changed: [localhost]

TASK: [Update apt cache] ******************************************************
ok: [localhost]

TASK: [Install Percona XtraDB Cluster server] *********************************
changed: [localhost] => (item=percona-xtradb-cluster-server-5.6,python-mysqldb,xinetd,telnet)

TASK: [Copy the helper script] ************************************************
changed: [localhost]

TASK: [Add Docker database user] **********************************************
changed: [localhost]

TASK: [Add xtrabackup database user (for Galera SST)] *************************
changed: [localhost]

TASK: [Add clustercheck database user (for clustercheck/xinetd -> haproxy)] ***
changed: [localhost]

TASK: [Configure Percona XtraDB Cluster server] *******************************
changed: [localhost]

TASK: [Stop MySQL] ************************************************************
ok: [localhost]

TASK: [Copy clustercheck script] **********************************************
changed: [localhost]

TASK: [Copy clustercheck script] **********************************************
changed: [localhost]


Verifying image

When this has completed, the image, capttofu/docker-dna_base, will be ready to use, in this case a container running Percona XtraDB Cluster that will need to be managed by Ansible in order to set up the galera cluster.

$ docker images
REPOSITORY                 TAG      IMAGE ID      CREATED       VIRTUAL SIZE
<none>                     <none>   cba0a737d934  14 hours ago  850.7 MB
ubuntu                     12.10    e314931015bd  13 days ago   172.2 MB
ubuntu                     quantal  e314931015bd  13 days ago   172.2 MB
ubuntu                     13.10    145762641db9  13 days ago   180.2 MB
ubuntu                     saucy    145762641db9  13 days ago   180.2 MB
ubuntu                     14.04    ad892dd21d60  13 days ago   275.5 MB
ubuntu                     latest   ad892dd21d60  13 days ago   275.5 MB
< ... snip ... >
capttofu/docker-dna_base   latest   167dc428d943  4 months ago  350.6 MB
capttofu/docker-dna_base   0.1.0    1e47da3640f1  4 months ago  798.7 MB
capttofu/docker-dna_base   12.04.w1 5150446a5dd3  4 months ago  350.6 MB


Summary

This blog post showed the reader yet another way to use Docker and Ansible together to build Docker images by using a Dockerfile to run Ansible to install packages and configure the temporary container that is being used to build the image. This provides yet another example of the flexibility of these two great applications and gives the user yet another method in their toolbox of solutions. Another side-benefit of this article was also learning how to install Percona XtraDB Cluster with Ansible.

comments powered by Disqus