Intent of this tutorial:

  • Use existing tools, i.e. don't roll our own networking stack.
  • Use vanilla OS images and change them as little as possible.
  • Keep things quick and clear.
  • Use plain LXC, libvirt and virsh don't work with LXC very well yet.

You can find my earlier attempt to get LXC 0.6 working at LXC/Lucid. Now that 0.7 has been released, it's time to update things.

It's not interesting that I'm doing this on Fedora 13 and the previous tutorial was on Ubuntu Lucid...  Either host should work fine.

Fedora 13 Warning

For now, don't bother using LXC on Fedora 13.  I can't get it to work well.  It seems close but I just can't figure out how to get lxc-console to connect.  My email to the LXC list has only produced people agreeing that F13 has problems!  Lucid is the only host I can recommend for now.

Installation

Fedora 13

Fedora now has 0.7.2 in the F13 repos.  Couldn't be easier!  Note that I haven't gotten F13 working well yet, see the start of this document for more.

 yum install lxc

Now skip to the Verify Install step.

Ubuntu Lucid

You can get reasonably recent lxc 0.7 packages from Serge Hallyn's PPA.

add-apt-repository ppa:serge-hallyn/virt
apt-get update
apt-get install lxc
apt-get install libvirt-bin

Now skip to the Verify Install step.

Install From Source

Install from source to get the very latest 0.7.x code with all bugfixes.

 rpm -e lxc lxc-libs        # if you installed the 0.6 packages 
yum install libcap-devel   # or apt-get install libcap-dev on Lucid
git clone git://lxc.git.sourceforge.net/gitroot/lxc/lxc
cd lxc
./autogen.sh
./configure
make

If you see nsgmls:(invalid location):E: invalid filename "" errors, don't worry about it.  It doesn't seem to affect anything.

Now install the binary and run lxc-setcap to allow non-root users to use containers.  Since I always manipulate containers as root in this tutorial, running lxc-setcap is recommended but strictly optional.  Don't worry about it if it fails.

sudo make install
sudo lxc-setcap

Verify Install

 lxc-checkconfig

Make sure that everything comes up green!  If you're using the stock Ubuntu or Fedora kernel this will almost certainly succeed.

Control Group Filesystem

Add the cgroup mount to your /etc/fstab file:

mkdir -p /cgroup
echo 'none /cgroup cgroup defaults 0 0 ' >> /etc/fstab
sudo mount /cgroup

libvirt Networking

Now we need to set up networking.  We'll use libvirt's default NATing setup because it's the easiest to administer: we don't need to worry about assigning IP addresses and routing.

 yum install libvirt        # apt-get install libvirt on Lucid
 /etc/init.d/libvirtd start

brctl show
    bridge name	bridge id		STP enabled	interfaces
    pan0		8000.000000000000	no		
    virbr0		8000.000000000000	yes

virsh net-list --all
    Name                 State      Autostart
    -----------------------------------------
    default              active     yes  
     
cat /proc/sys/net/ipv4/ip_forward 
    1

If your system produces similar output (a virbr0 interface, active, with ip_forwarding turned on), then libvirt's networking should be ready to go.

Configure Templates

Create the /etc/lxc-basic.conf file with these contents:

lxc.network.type=veth 
lxc.network.link=virbr0 
lxc.network.flags=up

This ensures that all the machines you create will use the libvirt networking without requiring any configuration.

Create a Machine

NOTE: We use lxc's built-in templates.  While they're pretty rough in the 0.7 release (very limited and don't offer any sort of help when they fail), scripts like lxc-debian, lxc-provider, and one-offs seem like dead-ends to me.  It would be nice to see these developres contributing to lxc's templates instead of rolling their own.  Creating guests by hand is still possible of course, but it's tedious and error-prone.  I described how to do this in the previous version of this tutorial, and so does Bodhi Zazen's site.  You might try any of these if you can't get LXC templates to work.

Now that you've installed lxc, it's time to create a machine.  (instructions provided by stlsaint)

yum install debootstrap   # or apt-get install debootstrap
sudo lxc-create -n lucid-guest -f /etc/lxc-basic.conf -t ubuntu

That creates a Lucid guest image in /usr/local/var/lib/lxc/lucid-guest.  It has same architecture as your host system (i.e. if you're running 64 bit, the guest will be 64 bit).  This is all hard-coded for now, hopefully the create scripts will get some attention soon.

NOTE: debootstrap creates a 153 MB image, lxc-create creates a 250MB image.  Why is lxc-create's so much larger?

The first time you run a create-ubuntu or create-debian script, it downloads all the packages over the network.  It keeps them cached in /var/cache/lxc however, so every time you run it after that things go pretty quick.

Fire it Up

Launch your new guest:

chmod a+w $(tty)    # TODO: this is unfortunate
lxc- start -n ubuntu-guest -o $(tty)  -l DEBUG

Your terminal fills with a lot of logging.  Hopefully it stops without bouncing you back to a shell prompt.  If so, then your machine should be running!  Check it out:

lxc-ps aux --n ame ubuntu-guest

If that produces output, you're in good shape!  If not, scroll back through the debugging messages to see what went wrong.

Now, let's log into our guest and start using it.  Open a new terminal since your existing one will be frozen until the guest exits (you can play with the -o and -d options to lxc-start to change that) and type:

lxc-attach -n ubuntu-guest
lxc-attach: No such file or directory - failed to open '/proc/25191/ns/pid'
lxc-attach: failed to enter the namespace

... and it fails.  Sure enough, /proc/25191/ns doesn't exist.  Why not?

Miscellany

lxc-attach doesn't work

The lxc-attach command won't work on any normal kernel.  It dies with the following error message:

 lxc-attach: No such file or directory - failed to open '/proc/12580/ns/pid'
lxc-attach: failed to enter the namespace 

That's OK because the lxc-attach command is not necessary for using LXC.  Applying these patches fixes it:

0006-ns-proc-files-for-namespace-naming-policy.patch
0007-ns-Introduce-the-setns-syscall.patch
0008-ns-proc-Add-support-for-the-network-namespace.patch
0009-ns-proc-Add-support-for-the-pid-namespace.patch
0010-ns-proc-Add-support-for-the-uts-namespace.patch
0011-ns-proc-Add-support-for-the-ipc-namespace.patch
0012-ns-proc-Add-support-for-the-mount-namespace.patch

You can find them here: http://lxc.sourceforge.net/patches/linux/  They're not in 2.6.35 or earlier, and I'm not sure when they'll be merged.

rsync is fine

Copying machines is easy.  You can use anything that will preserve device and special files.  Here's an rsync command that copies ubu-guest into root's home folder on the destination machine.

 sudo rsync -e 'ssh -c blowfish' --stats --delete -aHDx \
    --numeric-ids ubu-guest root@192.168.1.124: 

Note that you must run this command as root, and log in as root on the remote computer.  If not, ownership and device files will be broken.