Discovering SmartOS as a Home Hypervisor

       1811 words, 9 minutes

Doing some tidying in my network, computers and VPS collection, I went looking for a piece of software that could be used :

I (re)discovered SmartOS and decided to look at how it would fit or not. I keep my notes here.


Read about creating a SmartOS bootable USB key .

I simply copied the smartos-20231116T064739Z file to my Ventoy flash drive. Then inserted the installation media and booted from it.


The SmartOS Setup wizard starts automatically.

My specifics

There are a few non-standard things related to my installation. So I first drop to a shell, configure my stuff and then switch back to the installation process.

Press [enter] to continue^C

Bringing up a shell. When you are done in the shell hit ^D to return to the
system configuration tool.

[root@ /]# █

The default keyboard layout is US. I want to use the FR layout.

# kbd -s French
USB keyboard

My machine has 6 NICs and only one is active during installation. For some reasons, the interface numbering in the OS doesn’t match the one labelled on the machine. And it changes everytime I’m testing a new installation. The following commands helps me guess which one to use during installation.

# dladm show-phys
igb1  Ethernet  down   0      half    igb1
igb2  Ethernet  down   0      half    igb2
igb3  Ethernet  down   0      half    igb3
igb0  Ethernet  down   0      half    igb0
igb4  Ethernet  down   0      half    igb4
igb5  Ethernet  up     1000   full    igb5

Everything is now ready for the installer process.

# exit

Resuming the system configuration tool.

During my testings, I had to reset the ZFS disk configuration. It is not as simple as using dd on the first bits of disks. There seem to be ZFS information in the end of disks.

The simple method is to get a shell on SmartOS and issue the followind command:

# sdc-factoryreset
WARNING: This machine will reboot and destroy its ZFS pools after rebooting.
Do you want to proceed with the factory reset? (y/n) y<Enter>

This will destroy ALL DATA on the system, including potential customer data.
Are you sure? (y/n) y<Enter>

Rebooting in 5 seconds ... now!

SmartOS Setup

The installation process is as simple as “answer my questions”. Here are the answers I gave:

Launching a shell.
Please manually create/import a zpool named "zones".
# diskinfo
TYPE  DISK     VID      PID              SIZE         RMV SSD
SCSI  c1t0d0   SanDisk  Extreme Pro       119.25 GiB  yes no
SATA  c2t0d0   ATA      BT58SSD10M        238.47 GiB  no  yes
SATA  c2t1d0   Samsung  SSD 870 QVO 8TB  7452.04 GiB  no  yes

# zpool create -B -f zones c2t0d0
# zpool create    -f areas c2t1d0
# zpool list
areas 7.27T  102K 7.27T       -        -   0%  0% 1.00x ONLINE -
zones  238G  100K  238G       -        -   0%  0% 1.00x ONLINE -

# exit

Review the configuration and commit installation.

When the screen turns black, remove the installer media and watch SmartOS booting.

Basic configuration

Log in on the console or via SSH.

Localized keyboard

Have the console use the French keyboard layout.

# kbd -s French
# echo "default_keymap=france" >> /usbkey/config

Secured SSH connections

Register the SSH public key(s).

# mkdir -p /usbkey/

# echo ssh-ed25519 (...) > /usbkey/
# echo "root_authorized_keys_file=authorized_keys" >>/usbkey/config

Allow connection using SSH keys only.

# sed -i.orig \
-e 's/PermitRootLogin yes/PermitRootLogin prohibit-password/' \
-e 's/PasswordAuthentication yes/PasswordAuthentication no/'  \

# svcadm restart ssh

The Global Zone is non-persistent. SmartOS populates standard configuration files (/etc …) during boot using the usbkey directory content.

A reboot will make sure the system configured as expected.

Sending system emails

Configure email relay and credentials that can be used by the system to send electronic mails.

# vi /usbkey/config

Reboot to apply changes.

Change root password

When using US layout, I often use temporary and easy (were keys are identical in US and FR layouts) password. When I get an FR layout on the console or SSH access, I modify the root password.

The official documentation about changing the root password in the Global Zone makes it really easy.

# umount /etc/shadow
# cp /usbkey/shadow /etc/shadow
# passwd root
# cp /etc/shadow /usbkey/shadow

ZFS compression and deduplication

Enable ZFS compression on the data storage pool. I choose LZ4 because zfs(8) says:

The lz4 compression algorithm is a high-performance replacement for the lzjb algorithm. It features significantly faster compression and decompression, as well as a moderately higher compression ratio than lzjb, but can only be used on pools with the lz4_compress feature set to enabled. See zpool-features(7) for details on ZFS feature flags and the lz4_compress feature.

# zpool get feature@lz4_compress areas
areas  feature@lz4_compress  active  local

# zfs set compression=lz4 areas

# zfs get compression areas
areas  compression  lz4       local

# zfs get compression
NAME                PROPERTY     VALUE     SOURCE
areas               compression  lz4       local
zones               compression  off       default
zones/archive       compression  lzjb      local
zones/boot          compression  off       default
zones/config        compression  off       default
zones/cores         compression  gzip      local
zones/cores/global  compression  gzip      inherited from zones/cores
zones/dump          compression  off       local
zones/opt           compression  off       default
zones/swap          compression  off       default
zones/usbkey        compression  off       default
zones/var           compression  off       default

Deduplication is a great feature, when you read about it. I did test the feature years ago and was not convinced with its benefits. Another recent reading indicates than it may be an overlooked feature ; or at least highly depending on your usage and data. I will create a few datasets, enable dedup on them and will have the opportunity to remove it depending on what happens in time.

# for d in backup nocloud vms; do \
  zfs create areas/$d ;                    \
  zfs set dedup=edonr,verify areas/$d ;    \

# zfs get dedup,compression
NAME                PROPERTY     VALUE          SOURCE
areas               dedup        off            default
areas               compression  lz4            local
areas/backup        dedup        edonr,verify   local
areas/backup        compression  lz4            inherited from areas
areas/nocloud       dedup        edonr,verify   local
areas/nocloud       compression  lz4            inherited from areas
zones               dedup        off            default
zones               compression  off            default

Update SmartOS

Switching from one release to another seem quite easy. During my testing, I only did upgrades. And they were straightforward.

# piadm avail

# piadm install 20231005T000607Z zones
# piadm list
20230921T034751Z       zones/boot                     next       yes  yes
20231005T000607Z       zones/boot                     available  no   no

# piadm activate 20231005T000607Z zones
# piadm list
20230921T034751Z       zones/boot                     available  yes  no
20231005T000607Z       zones/boot                     next       no   yes

# reboot

Extra packages

SmartOS uses pkgsrc as a third-party package management. I’d rather not install much on the Global Zone. But a few basic tools might be handy.

# pkgin update
reading local summary...
processing local summary...
processing remote summary (

# pkgin in tmux
installing tmux-3.3anb2...
marking tmux-3.3anb2 as non auto-removable

SMB shares and Time Machine

One of the purpose of this NAS system is to serve as a backup system, for UNIX, Linux and macOS computers. Another is to provide access to files from various workstations ; the more common technology is Samba.

I don’t want to do it from the Global Zone. Let’s use a Brand Zone to do this.

Get a Zone image

Check what is available:

# imgadm avail name=base-64-lts
UUID                                  NAME         VERSION  OS       TYPE          PUB
c02a2044-c1bd-11e4-bd8c-dfc1db8b0182  base-64-lts  14.4.0   smartos  zone-dataset  2015-03-03
24648664-e50c-11e4-be23-0349d0a5f3cf  base-64-lts  14.4.1   smartos  zone-dataset  2015-04-17
b67492c2-055c-11e5-85d8-8b039ac981ec  base-64-lts  14.4.2   smartos  zone-dataset  2015-05-28
96bcddda-beb7-11e5-af20-a3fb54c8ae29  base-64-lts  15.4.0   smartos  zone-dataset  2016-01-19
088b97b0-e1a1-11e5-b895-9baa2086eb33  base-64-lts  15.4.1   smartos  zone-dataset  2016-03-04
1f32508c-e6e9-11e6-bc05-8fea9e979940  base-64-lts  16.4.1   smartos  zone-dataset  2017-01-30
390639d4-f146-11e7-9280-37ae5c6d53d4  base-64-lts  17.4.0   smartos  zone-dataset  2018-01-04
c193a558-1d63-11e9-97cf-97bb3ee5c14f  base-64-lts  18.4.0   smartos  zone-dataset  2019-01-21
e75c9d82-3156-11ea-9220-c7a6bb9f41b6  base-64-lts  19.4.0   smartos  zone-dataset  2020-01-07
1d05e788-5409-11eb-b12f-037bd7fee4ee  base-64-lts  20.4.0   smartos  zone-dataset  2021-01-11
c8715b60-7e98-11ec-82d1-03d16599f529  base-64-lts  21.4.0   smartos  zone-dataset  2022-01-26
85d0f826-0131-11ed-973d-2bfeef68011c  base-64-lts  21.4.1   smartos  zone-dataset  2022-07-11
93bdf06a-01ef-11ed-81ff-bf0efad842c7  base-64-lts  20.4.1   smartos  zone-dataset  2022-07-12
e44ed3e0-910b-11ed-a5d4-00151714048c  base-64-lts  22.4.0   smartos  zone-dataset  2023-01-10

Import the latest available release:

# imgadm import e44ed3e0-910b-11ed-a5d4-00151714048c
Importing e44ed3e0-910b-11ed-a5d4-00151714048c (base-64-lts@22.4.0) from ""
Gather image e44ed3e0-910b-11ed-a5d4-00151714048c ancestry
Must download and install 1 image (250.9 MiB)
Downloaded image e44ed3e0-910b-11ed-a5d4-00151714048c (250.9 MiB)
Imported image e44ed3e0-910b-11ed-a5d4-00151714048c (base-64-lts@22.4.0)

Configure the Zone image

The configuration is done using a JSON file.

# vim /areas/manifests/smb.json
  "zpool": "areas",
  "brand": "joyent",
  "image_uuid": "e44ed3e0-910b-11ed-a5d4-00151714048c",
  "alias": "smb",
  "hostname": "smb",
  "ram": 512,
  "quota": 6144,
  "resolvers": [ "", "" ],
  "nics": [
    { "nic_tag": "admin", "ips": [ "dhcp", "addrconf" ] }

Manage the Zone

Create the Zone:

# vmadm create -f /areas/manifests/smb.json

Check the Zone state:

# vmadm list
UUID                                  TYPE  RAM   STATE    ALIAS
13dc0e2e-a59d-42d2-941a-eb9f3ce42dc5  OS    1024  running  smb

Delete the Zone:

# vmadm delete 13dc0e2e-a59d-42d2-941a-eb9f3ce42dc5
Successfully deleted VM 13dc0e2e-a59d-42d2-941a-eb9f3ce42dc5

To be continued

From there, I faced a few features that do fit me well.

It is recommended to create a dedicated Zone for an SMB service . While this makes sense to not turn the Global Zone into a mess, it also makes you dedicate a dataset to a SMB Zone. Mr Jeffrey A Jenner explains how it can be made more admin-friendly . But I don’t like the idea of tweaking the storage and risking loosing the data if I delete the Zone.

The other thing I don’t like is how SmartOS and/or vadm expect the images and the zones to be deployed in the root of a zpool. Mixing the virtual machines, the data in a flat layout isn’t something I like to do.

I won’t dig further with SmartOS at the moment. I like the Illumos world and will continue looking at such distributions. Maybe I’ll get back to SmartOS if other options don’t suit me well either.

For the record, here’re a few more links that I used to test things but won’t write about, yet.