Best Practices for SAN Transport

For array-based storage, SAN transport is often the best performing choice for backups when running on a physical proxy. It is unavailable inside virtual machines, so use SCSI HotAdd instead on a virtual proxy.
SAN transport is not always the best choice for restores. It offers the best performance on thick disks, but the worst performance on thin disks, because of round trips through two disk manager APIs,
AllocateBlock
and
ClearLazyZero
. For thin disk restore, NBDSSL is usually faster. Changed Block Tracking (CBT) must be deactivated for SAN restores, and a snapshot must be taken before restore. SAN transport does not support writing to redo logs (child disks including linked clones and snapshots), only to base disks. Finally, SAN transport is not supported on vVol datastores.
Provided the disk LUN backing an NVMe-oF datastore is accessible from the physical machine running backups, SAN transport mode might be supported. However, VDDK searches only SCSI LUNs during initialization, so backup administrators would probably have to specify all NVMe-oF disks in
vixDiskLib.transport.san.allowList
(see Initialize Virtual Disk API).
Before vSphere 5.5, when writing to SAN during restore, disk size had to be a multiple of the underlying VMFS block size, otherwise the write to the last fraction of a disk would fail. This was fixed in the ESXi 5.5 release.
Programs that open a local virtual disk in SAN mode might be able to read (if the disk is empty) but writing will throw an error. Even if programs call
VixDiskLib_ConnextEx()
with NULL parameter to accept the default transport mode, SAN is selected as the preferred mode if SAN storage is connected to the ESXi host. VixDiskLib should, but does not, check SAN accessibility on open. With local disk, programs must explicitly request NBDSSL mode.
For a Windows Server 2008 and later proxy, set SAN policy to
onlineAll
. Set SAN disk to read-only except for restore. You can use the diskpart utility to clear the read-only flag. SAN policy varies by Windows Server 2008 edition. For Enterprise and Datacenter editions, the default Windows SAN policy is
offline
, which is unnecessary when vSphere mediates SAN storage.
For SAN transport, one major factor impacting performance is that the read buffer should be aligned with the sector size, currently 512. You can specify three parameters for
VixDiskLib_Read
: start sector, number of sectors to read, and the buffer to hold data. The proper read buffer size can be allocated using, for example,
_aligned_malloc
on Windows or
posix_memalign
on Linux. SAN mode performs best with about six concurrent streams per ESXi host; more than six streams usually results in slower total throughput.
VDDK 8.0.1 and later provide a cache file to assist SAN disk device discovery. Before doing a SAN mode backup, each instance of VDDK scans disks to determine the device name and VMFS ID of the LUN. In large datacenters with many SAN based disks, this had a significant negative performance impact. Partners requested a cache mechanism to reduce frequency of scans. As implemented, cache file
scsiDiskList.json
is created in the
tmpDirectory
specified by the VDDK configuration file. When VDDK fails to open a disk with cached LUN path, because disk was added or deleted since last scan, LUNs are rescanned and the JSON file overwritten. This cache mechanism operates without customer intervention.