My openstack-on-nspawn work

Hein-Pieter van Braam 65ced4aa66 Enable memcached for keystone 3 years ago b6e93cc63c Add initial version for 3 years ago faf5ba8520 Add a 'parent' feature to the image builder 3 years ago
images.yml 65ced4aa66 Enable memcached for keystone 3 years ago

OpenStack on systemd-nspawn

Notes and scripts so far

The basic idea

  • Run a modern Linux distribution with modern kernels and debug tools
  • Run the OpenStack software itself using the most tested software combinations

To this end we'll run Fedora 26 with CentOS7 containers running RDO packages.


We will use Puppet to manage the base OS as well as the Configuration for the containerized OpenStack components. So for instance the Keystone container contains:

  • CentOS7 (systemd, systemd-networkd)
  • OpenStack keystone (centos-release-openstack-newton provided)
  • HTTPD from CentOS7

The services in the container are all configured to start at boot (httpd in the case of keystone) while Puppet will manage /var/lib/machines/keystone/etc/keystone/keystone.conf From the host.

Container-side network configuration

# cat /var/lib/machines/keystone/etc/systemd/network/ 


Other container-side configuration

  • Ensure that pts/0 is in securetty
  • Ensure systemd-networkd is started at boot
  • Ensure a root password is set

This will just make sure everything 'just works'

Host-side nspawn configuration

# cat /etc/systemd/nspawn/keystone.nspawn 


This will forward the keystone ports from the host to the container without any further interference.

(PrivateUsers=no is required to allow httpd to install suexec, if we switch to another httpd we may be able to use private users)

Other host-side configuration

  • Ensure systemd-networkd is started

systemd-nspawn and network namespaces for neutron-l3,dhcp,lbaas-agent

We can't create network namespaces from inside nspawn containers as long as nspawn doesn't set the shareable flag on the rootfs of the container. So for Neutron we need a workaround.

We need to create network namespaces in the host system's mount namespace. We can actually enter the mount namespace of the host if we have a file descriptor into the hosts' mount namespace available. Normally this would not be the case but we can make sure we get access to it by bind mounting the /proc filesystem of the host into the container at a different location. We then just need to make sure that calls to 'ip' get intercepted and ran in the host mount namespace.

The neutron agents don't call '/sbin/ip' but just 'ip' so by modifying the service file for the agents to include a different path we can make it call our own ip without modifying /sbin/ip.

The wrapper script:

if [ "$1" == "netns" ]; then
  if [ "$2" == "add" ] || [ "$2" == "delete" ]; then
    nsenter --mount=/hostproc/1/ns/mnt /sbin/ip "$@"
    exit $?

/sbin/ip "$@"
exit $?

The nspawn file for the container:




Note that we really only need to run namespace creation and destruction in the host mount namespace. Exec and everything else can remain in the container.

Image building

Some small tooling for the creation of the nspawn container. Currently runs only on DNF based distros. See and images.yml