Automated OpenBSD deployment on vmd(8)

       1064 words, 5 minutes

Now that I have an OpenBSD server hosted in the Wild and capable of doing virtualization , I’ll migrate all my VM hosted on Synology Virtual Machine Manager. But even if the OpenBSD installer is straight forward, deploying tens of VM takes some time. So I set up an automated environment that provides fast and (nearly) finger-less deployment.

The hypervisor

Following directions from OpenBSD FAQ - Virtualization , the host was quickly up & running.

First of all, check the CPU compatibility:

# dmesg | egrep '(VMX/EPT|SVM/RVI)'
vmm0 at mainbus0: VMX/EPT

Then configure the virtual network. I went for the “Option 3” configuration: the host will be a router and basic services provider ; the VMs will stand on a dedicated non-routable network.

# grep -v '^#' /etc/sysctl.conf
net.inet.ip.forwarding=1

# cat /etc/hostname.vether0
inet 10.15.5.12 255.255.255.0
group vm
# sh /etc/netstart vether0

# cat /etc/hostname.bridge0
add vether0
group vm
# sh /etc/netstart bridge0

# cat /etc/vm.conf
(...)
switch "vswitch0" {
  interface bridge0
}
(...)
vm "template" {
  disable
  memory 512M
  disk "/dev/null"
  interface {
    group "vm"
    switch "vswitch0"
  }
}

The virtualization daemon is now ready to be run:

# rcctl enable vmd
# rcctl start vmd

Finally, allow the future VM to talk to each other and to the Internet:

# vi /etc/pf.conf
(...)
# VM can talk to the Internet
match out on egress from vether0:network to any nat-to (egress)
(...)
# VM can talk to each other
pass quick on vm label "vm vswitch"
(...)
# Block all by default
block log label "default block"
(...)
# Accessible Internet stuff
pass out proto udp to any port { domain, ntp }
pass out proto tcp to any \
  port { domain, ntp, whois, ssh, smtp, smtps, submission, http, https, imaps, 8080 } \
  flags S/SA modulate state \
  label "internet resources"

From there, create, install and launch VMs using the standard tools:

# vmctl create vm1.qcow2 -s 20G
# vmctl start vm1 -c -b /bsd.rd -d vm1.qcow2 -i 1 -m 256M -n vswitch0
# vmctl start vm1 -c -d vm1.qcow2 -i 1 -m 256M -n vswitch0

Boot and Install from the network

When the OpenBSD installer is netbooted, the autoinstall process starts after a short period of time. If the DHCP server provides the proper options, OpenBSD will autoinstall itself.

DHCP server

Configure dhcpd(8) so that the OpenBSD installer gets an IP address and knows where to fetch the autoinstall script:

# vi /etc/dhcpd.conf
subnet 10.15.5.0 netmask 255.255.255.0 {
  range 10.15.5.70 10.15.5.79;

  option domain-name "tumfatig.local";
  option option-119 "tumfatig.local tumfatig.net";
  option domain-name-servers 10.15.5.12, 8.8.8.8;

  option routers 10.15.5.12;

  # unattended OpenBSD installation and upgrade
  
  
  (...)
  # Testing servers
  group {
    default-lease-time 3600;
    max-lease-time 3600;
  }
}
# rcctl enable dhcpd
# rcctl set dhcpd flags vether0
# rcctl start dhcpd

HTTP server

Configure httpd(8) so that the OpenBSD installer can fetch an autoinstall script. I also have a local copy of the standard installation sets and I created a siteXX.tgz archive to customize the default server configuration:

# rcctl enable httpd
# vi /etc/httpd.conf
server "10.15.5.12" {
  listen on 10.15.5.12 port 80
  location "/pub/OpenBSD/6.5/amd64/*" {
    directory auto index
    root "/htdocs/openbsd-6.5-amd64"
    request strip 4
  }
  location "/*" {
    directory no index
    root "/htdocs/autoinstall"
  }
}
# rcctl start httpd

# cat /var/www/htdocs/autoinstall/install.conf
(...)
template for disklabel = http://10.15.5.12/disklabel.conf
System hostname = amnesia
(...)
Location of sets = http
HTTP proxy URL = none
HTTP Server = 10.15.5.12
Server directory = pub/OpenBSD/6.5/amd64
Use http instead = yes
Set name(s) = all
Set name(s) = done
Checksum test for site65.tgz failed. Continue = yes
Unverified sets: site65.tgz. Continue = yes
Location of sets = done

# cat /var/www/htdocs/autoinstall/disklabel.conf
/         10G-* 100%
swap       2G
/var/log   2G

Network installation using vmctl(8)

Using vmctl(8), the VM can boot from the network and the OpenBSD installer shall run in non-interactive mode:

# vmctl start test -c -B net -b /var/www/htdocs/openbsd-6.5-amd64/bsd.rd \
  -d test.qcow2 -t template
(...)
PXE boot MAC address fe:e1:bb:d1:7e:8b, interface vio0
root on rd0a swap on rd0b dump on rd0b
erase ^?, werase ^W, kill ^U, intr ^C, status ^T

Welcome to the OpenBSD/amd64 6.5 installation program.

(I)nstall, (U)pgrade, (A)utoinstall or (S)hell?
vio0: bound to 10.15.5.71 from 10.15.5.12 (fe:e1:ba:d0:55:5b)
Fetching http://10.15.5.12/fe:e1:bb:d1:7e:8b-install.conf?path=6.5/amd64
Fetching http://10.15.5.12/install.conf?path=6.5/amd64

(...)
CONGRATULATIONS! Your OpenBSD install has been successfully completed!

When you login to your new system the first time, please read your mail
using the 'mail' command.

syncing disks... done

From there, OpenBSD is installed on the disk file and the VM has been stopped. The VM can now be started using vmctl(8) or configured properly in vm.conf(5):

# vmctl start test -c -d test.qcow2 -t template
Connected to /dev/ttyp5 (speed 115200)
Using drive 0, partition 3.
Loading......
probing: pc0 com0 mem[638K 510M a20=on]
disk: hd0+
>> OpenBSD/amd64 BOOT 3.43
(...)
kern.timecounter.hardware: i8254 -> tsc
starting network
vio0: bound to 10.15.5.71 from 10.15.5.12 (fe:e1:ba:d0:55:5b)
reordering libraries: done.
openssl: generating isakmpd/iked RSA keys... done.
ssh-keygen: generating new host keys: RSA DSA ECDSA ED25519
starting early daemons: syslogd pflogd ntpd.
starting RPC daemons: portmap ypldap ypbind.
(...)
OpenBSD/amd64 (amnesia.tumfatig.local) (tty00)

login:

Customize the default installation

On the previous boot extract, one can see the kernel timecounter being configured to a non-default value and some extra daemons starting. The customization happens because there is a siteXX.tgz archive available from my installation repo. Here’s what it looks like:

# tar tvzpf site65.tgz
-rwxr-xr-x 1 root wheel 2180 Jun 19 15:14 install.site
-rwxr-xr-x 1 root wheel   50 Jun 19 15:07 upgrade.site

# cat install.site
#!/bin/ksh
# Enable softdep
sed -iorig -e "s/ffs rw,/ffs rw,softdep,/" /etc/fstab
# Enable tmpfs
echo 'swap /tmp mfs rw,nodev,nosuid,-s=64m 0 0' >> /etc/fstab
# Configure NTP
rcctl set ntpd flags -s
echo "kern.timecounter.hardware=tsc" >> /etc/sysctl.conf
# Disable audio server
rcctl disable sndiod
# Set mirror server location
echo "https://cdn.openbsd.org/pub/OpenBSD" > /etc/installurl
# Enable LDAP authentication
rcctl enable portmap ypldap ypbind
pkg_add login_ldap
cat >> /etc/login.conf << EOF
(...)
# Self-termination
exec rm /*.site

Conclusion

Popping a new ready-to-go OpenBSD instance now takes less than 5 minutes. As of OpenBSD 6.5, using netboot from vm.conf(5) is not possible . One will have to wait for the next release to be able to do it.

Bibliography & Credits

Related posts: