nx3d.org

LXD profiles and networking

Keywords:

LXD is a pretty powerful tool, but it is not always immediately obvious how to accomplish certain things such mixing static and dynamic IP address allocation. Many search results on this topic are regarding post-configuration of the container (via e.g. netplan) which is not exactly what we want: ideally, this configuration is managed on the LXD side.

Some background: LXC and LXD are a great platform for running Linux-based applications in a “realistic” environment. For example, at Hatching we test our virtual machine creation and management product using LXD. This enables us to test our product in an environment that is comparable to what a customer would run, so using this approach we hopefully catch deployment errors as well as software bugs.

In our case, our deployment environment is based on Ubuntu 18.04, with a number of services managed by systemd. Running virtual machines also requires access to KVM (hardware acceleration) and some Linux network internals (bridges, tap interfaces, ip route, iptables). We don’t really want to use nested virtualization, so testing all of this in a virtual machine is not an option.

To accomplish this, I’ve created a profile that describes the requirements of our environment:

name: vm-testing
description: VM testing container
config: {}
devices:
  # Bridged network with NAT
  eth0:
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic

  root:
    path: /
    pool: default
    type: disk

  # Example: share some resources from the host with the guest
  # (e.g. big ISOs)
  resources:
    source: /data/resources
    path: /usr/share/project/resources
    type: disk

  # Acceleration for QEMU
  kvm:
    path: /dev/kvm
    type: unix-char

  # QEMU uses tap devices for networking
  tun:
    path: /dev/net/tun
    type: unix-char

To create a container, you run:

lxc launch --profile vm-testing images:ubuntu/bionic/amd64 \
    mycontainer1

This assigns a dynamic IP address from the lxdbr0 pool, which is ideal for short-lived containers. We don’t have to manage IP address allocation ourselves, as it is managed by LXD and dnsmasq.

But what if you want to assign a static IP address to a container managed by LXD? For example, we also have containers that run the latest development version of our product, (permanently) reachable over the network. While the LXD documentation is not very clear on how to accomplish this, this is actually quite easy:

lxc init --profile vm-testing images:ubuntu/bionic/amd64 \
    mystatic1
lxc config device overide mystatic1 eth0 ipv4.address=10.3.4.5
lxc start mystatic1

You can either override specific configuration keys or even completely replace the eth0 device (by using lxc config device add with NIC with the same name) that would otherwise by specified by the profile. That’s it!