Migrate a FreeBSD bhyve virtual machine to OmniOS

       483 words, 3 minutes

I have a home lab running FreeBSD bhyve hypervisor and a collo server running OmniOS. From time to time, I migrate some VMs from one to the other. It is very simple when using ZFS and sharing a compatible bhyve implementation.

Create a new VM

Check the source (FreeBSD) virtual machine sizing parameters:

(freebsd)# vm info searxng
------------------------
Virtual Machine: searxng
------------------------
  state: running (15531)
  datastore: default
  loader: uefi
(...)
  cpu: 2
  memory: 1G
(...)
  network-interface
    number: 0
    emulation: virtio-net
(...)
  virtual-disk
    number: 0
    device-type: sparse-zvol
    emulation: virtio-blk
    system-path: /dev/zvol/tank/guests/searxng/disk0
    bytes-size: 17179869184 (16.000G)
    bytes-used: 2869633024 (2.672G)

Create the destination (OmniOS) virtual machine accordingly:

(omnios)# zadm create -b bhyve -t /zones/openbsd74.zadm searxng
{
   "autoboot" : "false",
   "bootdisk" : {
      "blocksize" : "8K",
      "path" : "tank/zones/searxng/root",
      "size" : "16G",
      "sparse" : "false"
   },
   "bootrom" : "BHYVE",
   "brand" : "bhyve",
   "diskif" : "virtio",
   "ip-type" : "exclusive",
   "net" : [
      {
         "global-nic" : "private0",
         "physical" : "searxng0"
      }
   ],
   "netif" : "virtio",
   "ram" : "1G",
   "rng" : "on",
   "type" : "generic",
   "vcpus" : "2",
   "vnc" : {
      "enabled" : "off"
   },
   "zonename" : "searxng",
   "zonepath" : "/zones/searxng"
}

CPU and RAM configuration don’t have to match. But those value are OK for me so I’ll keep using them. The Disk size should be the same to avoid weird things happening ; especially if the new disk is smaller than the original.

Transfer the VM disk volume

To avoid unexpected data corruption, the original VM is stopped before any data is transferred.

To limit the downtime on the source VM, I’ve created a snapshot of the stopped VM disk and restarted it.

(freebsd)# vm stop searxng
(freebsd)# zfs snapshot tank/guests/searxng/disk0@migrate
(freebsd)# vm start searxng

Now the service is up and running and I have no hurry to transfer the data. This also prevent stress mistakes.

On the OmniOS server, the dataset hosting the VMs is encrypted. On the FreeBSD, it is not. If I try to use ZFS to send the unencrypted volume to an encrypted dataset, the following error raises:

cannot receive new filesystem stream: zfs receive -F cannot be used to destroy an encrypted filesystem or overwrite an unencrypted one with an encrypted one

I simply deleted to destination ZFS volume and send the source one using the appropriate name.

(omnios)# zfs destroy tank/zones/searxng/root

(freebsd)# zfs send tank/guests/searxng/disk0@migrate | \
           ssh omnios.example "zfs recv tank/zones/searxng/root"

(omnios)# zfs list
NAME                      USED  AVAIL  REFER  MOUNTPOINT
(...)
tank/zones/searxng/root  3.99G  1017G  3.99G  -

Start the migrated VM

The network configuration is a bit different on the destination host. So I started the VM attaching the console to be able to boot in single user mode, apply the proper network and localtime modifications. Then proceed to normal boot.

(omnios)# zadm start -c searxng

That particular VM is running OpenBSD. But this should wor with any byhve compatible decent OS. The VM is now running in its new home.

That’s All Folks!

Related posts: