Hi,
for several days, we are trying to find a solution to improve performance when moving VM images from one (free)ESXi to another.
Our situation is as follows:
We have 4 different ESXi servers running, all hardware should be supported according to VMWares compatibility guide. Connected via gigabit network, all VMs stopped, nothing else running on the network. All VMs are stored locally on SATA disks.
Now we wanted to move a bunch of VM images from one ESXi to the others - but regardless of what technique we used, we achieved a max. speed of about 10 MByte/s (which would take several days to transfer everything).
We tried the following:
- Enable SSH and use SCP to copy the files directly between the servers (using several different encryption algorithms)
- Use Veeam FastSCP to copy the files directly
- Login via SSH on both sides and used a combination of tar&netcat to push raw files over the network
- Install ProFTPD and copy files via FTP
For all these methods, the write speed did never exceed ~10MByte/s (often it was even slower). So we thought the networking might have an issue, therefore we tried some local transfers.
But even a local "dd" to transfer data from 1 local disk to the other was just as slow as the network transfers... We tried this on all the different ESXi hardware we have, with RAID enabled, disabled, etc.
After some Googling, we found out that this should be an issue with the VMFS which doesn't really support the usage of "standard" filesystem tools. Therefore we tried to find a way to use ESX-internal tools to copy/clone the VM images.
We installed an NFS server "between" 2 ESXi servers, i.e. both have the NFS share accessible. Copying from ESXi to NFS is incredibly fast (via vSphere client, running at full gigabit speed) - but copying from NFS to ESXi is slow again (~10MByte/s as before). The final test we did was use the vmkfstools commandline utility to clone from the NFS share to the local VMFS disk. This seems to be a little bit faster (hard to measure due to missing statistics, seems to be about ~20MByte/s) but far from being a fast solution.
What is the expected way to move a VM image to a ESXi server? Is there a way to improve the transfer speed?