Moving VM on ESXi without VMotion or Storage VMotion

       1207 words, 6 minutes

One of the missing feature in the free version of ESXi 5 is the ability to move Virtual Machines between Datastores (Storage vMotion) and/or ESXi servers (vMotion). There is, however, a way to move a Virtual Machine from one hypervisor to another or to use another storage space ; at the cost of VM outage duration.

Here’s a brief review of what can be done and how regarding Virtual Machine instances and storage migration.

Moving a Virtual Machine to another location (Storage vMotion)

First of all, you’ll have to know that you can’t move a running Virtual Machine using the following directives. If you really need to do so, you’ll have to pay for the Storage vMotion feature. You can, however, move a paused or stopped Virtual Machine storage ; either totally or even a single part of it. Note that suspending a Virtual Machine requires that VMware Tools are installed.

You can choose to leave the configuration file on the initial Datastore and move the virtual disks to another. Or you can choose to move only one of its virtual disks to another Datastore.

The destination Datastore can either be dedicated to the ESXi or can be shared between ESXi hosts ; this will be used later on to move a VM from one host to another. The only requirement is that the ESXi servers can access the Datastore at the same time. I used a NFS share but I expect a VMFS on FC or iSCSI to work as well.

To move a Virtual Machine from one Datastore to another, you have to:

In my case, it took about 40min to move a 40GB set of files (from local storage to NFS shared storage).

While the move is in progress, you can have a look at the “Performance” tab of the ESXi server to see what’s going on. What I like to have a look at is rates vs latency.

Statistics on NFS (destination) datastore while moving Statistics on SSD (source) datastore while moving

When the move process is finished, don’t try to start the Virtual Machine right-away. You’ll get an error message saying “File (…).vmx was not found”. This is because all the files have been moved to the “destination” Datastore but the ESXi server still references them on the “source” Datastore.

To recover the Virtual Machine, using the vSphere client:

You can now start the Virtual Machine.

You’ll get a warning message telling you that “The Virtual Machine might have been moved or copied”. Choose “moved” as this will tell the ESXi server to keep the internal IDs. You can use this procedure to clone a Virtual Machine ; in such a case, you’d have to select the “copied” state so that another ID set is generated on the ESXi server.

If you’re a lucky Administrator, the Virtual Machine should now be up and running from the “destination” Datastore.

Note that this moving process has few, if not none, failure management. During one of my testings, the move process stopped in the middle of the operation and I ended with some files splited on both Datastores. I could move the rest of the files in a second move operation and the VM was started without issue ; but in the case of several VM move, this can hurt a bit more.

Moving a Virtual Machine to another ESXi (vMotion)

This process will require that, at least temporary, the “source” ESXi server has access to some storage that will be used by the “destination” ESXi server. In my case, I use a NFS shared Datastore that is visible from both ESXi ; I expect a VMFS on FC or iSCSI to work as well.

As seen previously, you can’t migrate a “living” Virtual Machine ; it has to be paused or shutdown before it can be moved. This is quite expectable as there would be no reasons to pay for vMotion if it could be done by some other way…

What has to be know while dealing with ESXi migration is that, although we’re using virtual hardware, a running Virtual Machine has some kind of adherence to the physical CPU it is running on. In my case, I moved the VMs from a physical ESXi to a virtual one and the paused Virtual Machines couldn’t be restarted in their paused state. You can probably face the same problem if migrating between ESXi that have not the exact same CPU (and CPU features). The error I got was “Your virtual machine did not resume because of a correctable error”. I choose the “Delete” option and the VM started as if I had been savagely powered down. So my advice is to properly shutdown the Virtual Machine before migrating it from one ESXi server to another.

The migration process is then really simple ; at least as simple as seen in the previous section. We’ll assume that the Virtual Machine is already located in some shared space.:

You can now start the Virtual Machine.

As seen previously, you’ll get a warning message telling you that “The Virtual Machine might have been moved or copied”.

If you’re a lucky Administrator, the Virtual Machine should now be up and running from the “destination” ESXi server. Note that it only takes a few seconds to migrate a Virtual Machine from one ESXi to another (since the VMs is powerred off).

Conclusion

Although you can’t migrate a living Virtual Machine (Storage) from one Datastore/Hypervisor without the (Storage) vMotion features, it is still possible to do it in paused or shutdown mode.

Using a shared storage, it only takes a couple of seconds to move a Virtual Machine from one hypervisor to another. It’s quite longer to move the storage from one place to another.

If your Service Level Agreement can handle the outage duration of such operations, then you can still use the free version only of the ESXi hypervisor.

Related posts: