Debian Virtualization: LXC Application Containers

Linux containers (LXC) is a lightweight virtualization technology built into Linux kernel. Unlike other similar technologies, the virtual machines (VM) are driven without any overhead by the kernel already running on the computer. In VM you run only the processes you need, most of the time without even going through the regular SysV or Linux init. This means that memory is used very conservatively. These lightweight containers are sometimes called application containers, as opposed to distribution containers where you run a full distro starting with SysV/Linux init.

In this article I’ll take a look at installing and building a small web server application container using LXC on a Debian 7 workstation. Debian is a popular distribution for personal use among software developers, and the amount of software packaged far exceeds any other Linux disto out there. It also encourages tinkering, experimenting, and in a more general sense, learning about Linux and Free/Open Source Software.

The computer I am using is an old amd64 dual-core computer with a minimal LXDE desktop manager installed. For those of you interested, I have detailed the installation steps in my Lightweight Debian: LXDE Desktop From Scratch article. Since we are dealing with kernel utilities, all the commands in this article are specified as user root.

Installing LXC and activating Linux Control Groups (CGROUPS)

By default, a minimal Debian install doesn’t bring in LXC user-space tools. Also, CGROUPS have to be installed and enabled. The installation is as follows:

# apt-get install cgroup-bin libcgroup1 lxc

CGROUPS is a feature of the modern Linux kernels that allows the allocation of resources such as CPU time, system memory, network bandwidth among the various processes running on the system. To activate CGROUPS you have to add the following line in /etc/fstab file:

cgroup  /sys/fs/cgroup  cgroup  defaults  0   0

Reboot the computer and check LXC and CGROUPS installation:

# lxc-checkconfig
Kernel config /proc/config.gz not found, looking in other places...
Found kernel config file /boot/config-3.2.0-4-amd64
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances: enabled

--- Control groups ---
Cgroup: enabled
Cgroup clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled

--- Misc ---
Veth pair device: enabled
Macvlan: enabled
Vlan: enabled
File capabilities: enabled

Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig

Creating a file system for your VM

We start by creating a new vm1 directory with a basic filesystem skeleton, with the intention to populate it by mounting read-only the main software directories from the host computer. The resulting filesystem has a very small footprint:

# cd ~
# mkdir -p vm1/rootfs
# cd vm1/rootfs
# mkdir bin
# mkdir home
# mkdir lib
# mkdir lib64
# mkdir opt
# mkdir proc
# mkdir root
# mkdir sbin
# mkdir sys
# mkdir tmp
# mkdir usr
# mkdir var

We also need to create and populate /dev and /etc directories:

# cp -a /etc .
# cp -a /dev .
# rm -fr dev/pts
# mkdir dev/pts
# rm -fr dev/shm
# mkdir dev/shm

/etc can be further cleaned up of unnecessary files, user entries can be removed from /etc/passwd and /etc/shadow files, /home directory can also be set etc.

Creating LXC configuration file

Back in vm1 directory, next step is to create our virtual machine configuration file lxc.conf:

# cd ~/vm1
# cat lxc.conf
lxc.utsname = vm1
lxc.rootfs = vm1/rootfs
lxc.mount.entry=/lib lib none ro,bind 0 0
lxc.mount.entry=/bin bin none ro,bind 0 0
lxc.mount.entry=/usr usr none ro,bind 0 0
lxc.mount.entry=/sbin sbin none ro,bind 0 0
lxc.mount.entry=/lib64 lib64 none ro,bind 0 0
lxc.mount.entry=proc /proc proc nodev,noexec,nosuid 0 0
lxc.mount.entry=tmpfs /dev/shm tmpfs  defaults 0 0
lxc.pts=1024

It is a minimal configuration mounting read-only a big part of the real filesystem. The configuration doesn’t touch networking, the container in this example uses the same networking stack as the host computer.

Starting the virtual machine

Use lxc-execute command to start the VM and run applications in the isolated environment. When playing as root with this kind of VMs, it is a good idea to have the terminal running the virtual machine visibly different than your regular terminal. For example if you run an LXDE desktop environment and lxterminal, start your virtual machines in xterm (apt-get install xterm). This terminal is the controlling terminal for your virtual machine.

I open an xterm, su as root and start the virtual machine. Once started, I reset the terminal and go for a more descriptive prompt:

# cd ~
# lxc-execute -n vm1 -f vm1/lxc.conf -- /bin/bash
root@vm1:/# reset
root@vm1:/# export PS1="\e[01;31m\h:\W \u\$ \e[00m"
LXC virtual machine running in an xterm

LXC virtual machine running in an xterm

To shut down the virtual machine close the controlling terminal, or type exit several times. Before that, however, you would need to shut down or kill -9 all the processes you started manually in VM.

Installing and starting your application in VM

I use Lighttpd web server as an example. I installed it on my computer prior to starting the virtual machine as follows:

# apt-get install lighttpd

In Debian, Lighttpd is started by default when you install it. It needs to be stopped and removed from the bootup sequence. The files are served from /var/www and the configuration is in /etc/lighttpd/lighttpd.conf. We need to make sure these files are copied in our VM filesystem along with other thttpd-specific directories under /var:

# update-rc.d -f lighttpd remove
# /etc/init.d/lighttpd stop
# cp -a /var/www ~/vm1/rootfs/var/.
# mkdir ~/vm1/rootfs/var/log
# cp -a /var/log/lighttpd ~/vm1/rootfs/var/log/.
# mkdir -p ~/vm1/rootfs/var/cache
# cp -a /var/cache/lighttpd ~/vm1/rootfs/var/cache/.

In the virtual machine, start Lighttpd as /etc/init.d/lighttpd start.

Starting the virtual machine at boot time

The last thing to do is to have the virtual machine started at boot time, if you so desire. In this case using xterm as the controlling terminal will not work. I will turn my attention to GNU Screen (apt-get install screen).

GNU Screen is a software application that can be used to multiplex several virtual consoles inside a single terminal window or remote terminal session. I run lxc-execute through screen, and I inject /etc/init.d/lighttpd start in the screen session together with the terminal reset. For simplicity, all the commands go into /etc/rc.local file:

#!/bin/sh -e
#
# /etc/rc.local file starting vm1
#

cd /root
screen -AmdS vm1 bash
sleep 1
screen -S vm1 -p 0 -X stuff "/usr/bin/lxc-execute -n vm1 -f vm1/lxc.conf -- /bin/bash"
screen -S vm1 -p 0 -X stuff '\r'
sleep 1
screen -S vm1 -p 0 -X stuff "reset"
screen -S vm1 -p 0 -X stuff '\r'
sleep 1
screen -S vm1 -p 0 -X stuff "/etc/init.d/lighttpd start"
screen -S vm1 -p 0 -X stuff '\r'

exit 0

To connect to the virtual machine console use screen -R vm1 as root. This is how the default Lighttpd page should look in your web browser:

Default Lighttpd page

Default Lighttpd page

Conclusion

In this article I've tried to describe the main steps for building a webserver application container using LXC and a bunch of other free software tools. The distribution I used for this example is Debian 7 "wheezy", however the example should work on most Linux distributions out there. Most distros have LXC and CGROUPS support compiled by default in the kernel, and installing the user-space utilities is just a matter of grabbing them with your favorite package manager.

I didn't mention anything regarding security. At the current stage, LXC kernel implementation does not promise anything. As the implementation matures and new features are added, security features will start coming in. For now, think about LXC as a chroot technology on steroids, with all the problems chroot has. To add the necessary layer of security needed to run it in production, LXC is often used in conjunction with SELinux, AppArmor and other mandatory access control technologies.

The virtual machine described in this article uses the same TCP/IP network stack as the host. In case the VM is compromised, the intruder gains full access to the networks connected to the host. Typically, this is mitigated by isolating the VM on a different network segment, with a different TCP/IP network stack inside the virtual machine. This is a very common security measure in virtualization world. In my second article in this series, Debian Virtualization: LXC Network Isolation, I modify the VM to run its own separate networking stack, and move it out on a different network segment. Access to and from VM is then restricted using netfilter firewall in Linux kernel.

This is the script for VM creation, based on the instructions in this article:

#!/bin/bash
# build an LXC container based on your current filesystem
# mounted read-only 

if [ $# -ne 1 ]
then
	echo "Error: directory missing"
	echo "Usage: $0 directory-name"
	exit 1
fi

# create rootfs		
mkdir -p $1/rootfs
cd $1/rootfs
mkdir bin
mkdir home
mkdir lib
mkdir lib64
mkdir opt
mkdir proc
mkdir root
mkdir sbin
mkdir sys
mkdir tmp
mkdir usr
mkdir var
cp -a /etc .
cp -a /dev .
rm -fr dev/pts
mkdir dev/pts
rm -fr dev/shm
mkdir dev/shm
cd ..

# create lxc configuration
cat > lxc.conf << EOF
lxc.utsname = $1
lxc.rootfs = $1/rootfs
lxc.mount.entry=/lib lib none ro,bind 0 0
lxc.mount.entry=/bin bin none ro,bind 0 0
lxc.mount.entry=/usr usr none ro,bind 0 0
lxc.mount.entry=/sbin sbin none ro,bind 0 0
lxc.mount.entry=/lib64 lib64 none ro,bind 0 0
lxc.mount.entry=proc /proc proc nodev,noexec,nosuid 0 0
lxc.mount.entry=tmpfs /dev/shm tmpfs  defaults 0 0
lxc.pts=1024
EOF

# print startup instruction
cd ..
PWD=`pwd`
echo "Virtual machine created, start it as:"
echo "     # cd $PWD"
echo "     # lxc-execute -n $1 -f $1/lxc.conf -- /bin/bash"
echo

Save it as vmscript.sh, and chmod +x vmscript.sh. Run it as:

# cd ~
# /path/to/vmscript.sh vm1

Related Posts

About these ads

3 thoughts on “Debian Virtualization: LXC Application Containers

  1. Pingback: Links 12/8/2013: Netrunner 13.06, New Sabayon | Techrights

  2. Pingback: Debian Virtualization: LXC Application Containers | Hallow Demon

  3. Pingback: Container-based virtualization on Wheezy | 0ddn1x: tricks with *nix

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s