Sizing vSAN Resources for the Management Domain

Sizing vSAN Storage

You size your vSAN datastore according to the requirements of your organization. You size the datastore for a certain node configuration as according to your hardware specifications.

Although vSAN uses thin provisioning for virtual disks, to conserve capacity consumption and prevent sudden exhaustion of the vSAN capacity in the management domain, consider the full disk space in the calculations, that is, the space for thick provisioning.

Disk space usage distribution consists of the following components:

Component	Description
Effective Raw Capacity	Space available for the vSAN datastore
Slack Space	Space reserved for vSAN specific operations such as resync and rebuilds
Dedupe Overhead	Space reserved for deduplication and compression metadata such as hash, translation, and allocations maps
Disk Formatting Overhead	Reservation for file system metadata
Checksum Overhead	Space used for checksum information
Physical Reservation	Physical space or raw capacity consumed by the overheads

Sizing the vSAN Datastore for an Environment with a Single VMware Cloud Foundation Instance

When sizing your vSAN datastore, consider the required storage capacity for all management components running in the management domain for both the management domain and for the VI workload domains you plan to deploy. Consider also the additional capacity needed for file system and any third-party management components you want to add.

Hardware Resource Specifications of the Management Components in a Single VMware Cloud Foundation Instance
Category	Quantity	Resource Type	Consumption (GB)
Physical infrastructure (ESXi)	4	Memory	512
Management domain vCenter Server	1	Disk	694
		Swap	19
SDDC Manager	1	Disk	816
		Swap	16
Management domain NSX Manager Node	3	Disk	300
		Swap	24
Management domain NSX Edge Node	2	Disk	200
		Swap	8
VI workload domain vCenter Server	1	Disk	920
		Swap	28
VI workload domain NSX Manager Node	3	Disk	300
		Swap	48
Total	11 management virtual machines 4 ESXi hosts	Disk	4630
		Swap	295
		Memory	512

Design Decisions on Storage I/O Controller Configuration for vSAN
Decision ID	Design Decision	Design Justification	Design Implication
VCF-MGMT-VSAN-CFG-001	Ensure that the storage I/O controller that is running the vSAN disk groups is capable and has a minimum queue depth of 256 set.	Storage controllers with lower queue depths can cause performance and stability problems when running vSAN. vSAN ReadyNode servers are configured with the right queue depths for vSAN.	Limits the number of compatible I/O controllers that can be used for storage.
VCF-MGMT-VSAN-CFG-002	Do not use the storage I/O controllers that are running vSAN disk groups for another purpose.	Running non-vSAN disks, for example, VMFS, on a storage I/O controller that is running a vSAN disk group can impact vSAN performance.	If non-vSAN disks are required in ESXi hosts, you must have an additional storage I/O controller in the host.

Sizing the vSAN Datastore for an Environment with Multiple VMware Cloud Foundation Instances

In addition to the appliances you have deployed for a single VMware Cloud Foundation instance, accommodate the 3-node NSX Global Manager clusters that enable NSX Federation in the management domain and in the VI workload domains in each instance. See Logical Design for NSX-T Data Center for the Management Domain and Deployment Model for NSX Global Manager for the Management Domain.

Hardware Resource Specifications of the Management Components in a Dual-Instance Environment
Category	Quantity	Resource Type	Consumption (GB)
ESXi	4	Memory	1024
Management domain vCenter Server	1	Disk	694
		Swap	19
SDDC Manager	1	Disk	816
		Swap	16
NSX Local Manager node for the management domain	3	Disk	300
		Swap	24
NSX Global Manager node for the management domain	3	Disk	300
		Swap	24
NSX Edge node for the management domain	2	Disk	200
		Swap	8
VI workload domain vCenter Server	1	Disk	920
		Swap	28
NSX Manager node for a VI workload domain	3	Disk	300
		Swap	48
NSX Global Manager node for a VI workload domain	3	Disk	300
		Swap	48
Total	17 management virtual machines 4 ESXi hosts	Disk	6430
		Swap	511
		Memory	1024

Design Decisions on the vSAN Datastore for Multiple VMware Cloud Foundation Instances
Decision ID	Design Decision	Design Justification	Design Implication
VCF-MGMT-VSAN-CFG-006	Provide the default cluster in the management with a minimum of 19.86 TB of raw capacity for vSAN.	The management virtual machines require at least 6.36 TB of raw storage (before setting FTT to 1) and 12.73 TB when using the default vSAN storage policy. By allocating at least 19.86 TB, initially 30% of the space is reserved for vSAN internal operations and 20% of the space is free which you can use for additional growth of management virtual machines. NFS is used as secondary shared storage for some management components, for example, for backups and log archives.	If you scale the environment out with more workloads, additional storage is required in the management domain.

Number of vSAN-Enabled Hosts per Cluster

The number of ESXi hosts in the cluster depends on these factors:

The amount of available space on the vSAN datastore

The number of failures you can tolerate in the cluster

For example, if the vSAN cluster has 3 ESXi hosts, only a single failure is supported. For a higher level of availability, you must add more hosts.

Design Decision on the vSAN Cluster Size in a Management Domain with a Single Availability Zone
Decision ID	Design Decision	Design Justification	Design Implication
VCF-MGMT-VSAN-CFG-007	The default management cluster requires a minimum of 4 ESXi hosts to support vSAN.	Having 4 ESXi hosts addresses the availability and sizing requirements. You can take an ESXi host offline for maintenance or upgrades without impacting the overall vSAN cluster health.	The availability requirements for the management cluster might cause underutilization of the cluster's ESXi hosts.

Design Decision on the vSAN Cluster Size in a Management Domain with Multiple Availability Zones
Decision ID	Design Decision	Design Justification	Design Implication
VCF-MGMT-VSAN-CFG-008	To support a vSAN stretched cluster, the default management cluster requires a minimum of 8 ESXi hosts (4 in each availability zone) .	Having 8 ESXi hosts addresses the availability and sizing requirements. You can take an availability zone offline for maintenance or upgrades without impacting the overall vSAN cluster health.	The capacity of the additional 4 hosts is not added to capacity of the cluster. They are only used to provide additional availability.

Number of vSAN Disk Groups per ESXi Host

Disk group sizing is important during volume design. The number of disk groups can affect availability and performance. The more ESXi hosts are available in the cluster, the more failures are tolerated in the cluster.

In vSAN, each ESXi host can have up to five disk groups, each containing seven capacity devices, resulting in a maximum of 35 capacity devices. The optimal number of disk groups is a balance between hardware and space requirements for the vSAN datastore.

Consider these data points when deciding on the number of disk groups per ESXi host:

The amount of available space on the vSAN datastore

The number of failures you can tolerate in the cluster

The performance required when recovering vSAN objects

This capability adds cost because you must provide additional hardware for the disk groups. More available disk groups can decrease the recoverability time of vSAN during a failure too.

Design Decisions on the vSAN Disk Groups per ESXi Host
Decision ID	Design Decision	Design Justification	Design Implication
VCF-MGMT-VSAN-CFG-009	Configure vSAN with a minimum of two disk groups per ESXi host.	Reduces the size of the fault domain and spreads the I/O load over more disks for better performance.	Multiple disks groups require more disks in each ESXi host.

Sizing vSAN Disks per ESXi Host for an Environment with a Single VMware Cloud Foundation Instance

You size the vSAN disks according to the requirements for the datastore, the number of ESXi hosts in the vSAN cluster, and the number of disk groups per host.

For a single VMware Cloud Foundation instance, the required vSAN datastorse size is 37 TB of raw data. For sizing the vSAN caching tier, see the Considerations for Flash Caching Devices in vSAN post on VMware Blogs. For vSAN memory consumption by management ESXi hosts, see VMware Knowledge Base article 2113954.

Design Decisions on the vSAN Disk Configuration for a Management Domain for a Single VMware Cloud Foundation Instance
Decision ID	Design Decision	Design Justification	Design Implication
VCF-MGMT-VSAN-CFG-010	For the caching tier in each disk group, use a flash-based drive that is at least 600 GB large.	Provides enough cache for both hybrid or all-flash vSAN configurations to buffer I/O and ensure disk group performance. Additional space in the cache tier does not increase performance.	Larger flash disks can increase initial host cost
VCF-MGMT-VSAN-CFG-011	Allocate at least 2.3 TB of flash-based drives for the capacity tier in each disk group.	Provides enough capacity for the management virtual machines with a minimum of 30% of overhead and 20% growth when the number of primary failures to tolerate is 1.	None.

Sizing vSAN Disks per ESXi Host for an Environment with Multiple VMware Cloud Foundation Instances

For multiple VMware Cloud Foundation instances in your environment, the required vSAN datastorse size is 43 TB of raw data.

Design Decisions on the vSAN Disk Configuration for a Management Domain for Multiple VMware Cloud Foundation Instances
Decision ID	Design Decision	Design Justification	Design Implication
VCF-MGMT-VSAN-CFG-012	Have at least 3.31 TB of flash-based drives for the capacity tier in each disk group.	Provides enough capacity for the management virtual machines with a minimum of 30% of overhead and 20% growth when the number of primary failures to tolerate is 1.	None.

VMware Cloud Foundation 4.5