Shut Down a Virtual Infrastructure Workload Domain with vSphere with Tanzu

You shut down the components of a VI workload domain that runs containerized workloads in
VMware Cloud Foundation
in a specific order to keep components operational by maintaining the necessary infrastructure, networking, and management services as long as possible before shutdown.
You shut down the management components for the VI workload domains that run vSphere with Tanzu and containers or that run virtualized workloads before you shut down the components for the management domain.
If the NSX Manager cluster and NSX Edge cluster are shared with other VI workload domains, follow this general order:
  1. Shut down the customer workloads in all VI workload domains that share the NSX-T Data Center instance. Otherwise, all NSX networking services in the customer workloads will be interrupted when you shut down NSX-T Data Center.
  2. Shut down the VI workload domain that runs the shared NSX Edge nodes.
  3. Shut down the other VI workload domains.

Shutdown Order for a VI Workload Domain with vSphere with Tanzu

Shutdown Order for a VI Workload Domain with vSphere with Tanzu
Shutdown Order
SDDC Component
1
Containerized customer workloads
2
Find out the location of the vSphere with Tanzu virtual machines *
3
vSphere Cluster Services virtual machines in the VI workload domain *
4
vCenter Server for the VI workload domain *
5
6
7
8
9
NSX Edge nodes in the VI workload domain *
10
NSX Manager nodes for the VI workload domain *
11
VxRail Manager *
12
vSAN and ESXi hosts in the VI workload domain *
* For information on the shutdown steps, see below.

Find Out the Location of the vSphere with Tanzu Virtual Machines on the ESXi Hosts

Before you begin shutting down a VI workload domain with vSphere with Tanzu, you get a mapping between virtual machines in the workload domain and the ESXi hosts on which they are deployed. You later use this mapping to log in to specific ESXi hosts and shut down specific management virtual machines.
  1. Start Windows PowerShell.
  2. Connect to the VI workload domain vCenter Server by running the command.
    Connect-VIServer -Server <
    workload_domain_vCenter_server_fqdn
    > -User administrator@vsphere.local -Password
    vsphere_admin_password
  3. Generate the virtual machine to host mapping in a
    C:\VMToHostMapping.csv
    file on the Windows machine by running the command.
    Get-VM | Select Name,VMHost | Export-Csv -Path C:\VMToHostMapping.csv -NoTypeInformation

Shut Down the vSphere Cluster Services Virtual Machines

To shut down the vSphere Cluster Services (vCLS) virtual machines in a cluster in a VI workload domain in
VMware Cloud Foundation
, you put the cluster in retreat mode. The retreat mode triggers clean-up of the vCLS virtual machines.
  1. Log in to vCenter Server for the management or VI workload domain at
    https://<vcenter_server_fqdn>/ui
    as
    administrator@vsphere.local
    .
  2. In the
    Hosts and clusters
    inventory, expand the tree of the VI workload domain vCenter Server and expand the data center for the VI workload domain.
  3. Select the cluster on which vCLS must be shut down.
  4. Copy the cluster domain ID
    domain-c(
    cluster_domain_id
    )
    from the URL of the browser.
    When you navigate to a cluster in the vSphere client, the URL is similar to this one:
    https://<fqdn-of-vCenter-server>/ui/app/cluster;nav=h/urn:vmomi:ClusterComputeResource:domain-c
    8
    :eef257af-fa50-455a-af7a-6899324fabe6/summary
    You copy only
    domain-c8
    .
  5. In the
    Host and Clusters
    inventory, select the vCenter Server instance for the VI workload domain and click the
    Configure
    tab.
  6. Under
    Advanced Settings
    , click the
    Edit Settings
    button.
  7. Locate the
    config.vcls.clusters.domain-c(
    number
    ).enabled
    property for the domain cluster ID from shut-down-the-vsphere-cluster-services-virtual-machines-in-the-management-domain.dita#GUID-E1CA90D6-0330-4109-A4AF-AAD2D6C7ADBE-en_step_bdc8d04e-140c-4ee7-a6b4-536577523e7f and set it to
    false
    .
    If the property is not present, add it. The entry for the cluster cannot be deleted from the vSphere Client then. However, keeping this entry is not an issue.
  8. Click
    Save
    .
The vCLS monitoring service initiates the clean-up of vCLS VMs. If vSphere DRS is activated for the cluster, it stops working and you see an additional warning in the cluster summary. vSphere DRS remains deactivated until vCLS is re-activated on this cluster.

Shut Down vCenter Server for a Virtual Infrastructure Workload Domain with vSphere With Tanzu

To shut down the vCenter Server instance for a VI workload domain with vSphere with Tanzu in
VMware Cloud Foundation
, you use the vSphere Client. You stop the Kubernetes services and check the vSAN health status.
  1. Shut down the Kubertenes services on vCenter Server.
    1. Log in to vCenter Server as
      root
      by using a Secure Shell (SSH) client.
    2. To switch to the Bash shell, run the
      shell
      command.
    3. Stop the Kubernetes services by running the command.
      vmon-cli -k wcp
    4. Verify the Kubernetes services status by running the command.
      vmon-cli -s wcp
      The output must contain
      RunState: STOPPED
      .
  2. Log in to the management domain vCenter Server at
    https://<vcenter_server_fqdn>/ui
    as
    administrator@vsphere.local
    .
  3. Verify the vSAN health and resynchronization status.
    1. Select the vSAN cluster and click the
      Monitor
      tab.
    2. In the left pane, navigate to
      vSAN
      Skyline health
      and verify the status of each vSAN health check category.
    3. In the left pane, under
      vSAN
      Resyncing objects
      , verify that all synchronization tasks are complete.
  4. If a vSAN cluster in the workload domain has vSphere HA turned on, stop vSphere HA to avoid vSphere HA initiated migrations of virtual machines after vSAN is partitioned during the shutdown process.
    1. Select the vSAN cluster and click the
      Configure
      tab.
    2. In the left pane, select
      Services
      vSphere Availability
      and click the
      Edit
      button.
    3. In the
      Edit Cluster Settings
      dialog box, turn off vSphere HA and click
      OK
      .
    This operation takes several minutes to complete.
  5. In the
    VMs and templates
    inventory, expand the management domain vCenter Server tree and expand the management domain data center.
  6. Shut down vCenter Server.
    1. Locate the vCenter Server virtual machine for the VI workload domain.
    2. Right-click the virtual machine and select
      Power
      Shut down Guest OS
      .
    3. In the confirmation dialog box, click
      Yes
      .
    This operation takes several minutes to complete.

Shut Down the NSX Edge Nodes for vSphere with Tanzu

You begin shutting down the NSX-T Data Center infrastructure in a VI workload domain with vSphere with Tanzu by shutting down the NSX Edge nodes that provide north-south traffic connectivity between the physical data center networks and the NSX SDN networks.
Because the vCenter Server instance for the domain is already down, you shut down the NSX Edge nodes from the ESXi hosts where they are running.
  1. Log in to the ESXi host that runs the first NSX Edge node as
    root
    by using the VMware Host Client.
  2. In the navigation pane, click
    Virtual machines
    .
  3. Right-click an NSX Edge virtual machine, and select
    Guest OS
    Shut down
  4. In the confirmation dialog box, click
    Yes
    .
  5. Repeat these steps to shut down the remaining NSX Edge nodes for the VI workload domain with vSphere with Tanzu.

Shut Down the NSX Manager Nodes

You continue shutting down the NSX-T Data Center infrastructure in the management domain and a VI workload domain by shutting down the three-node NSX Manager cluster by using the vSphere Client.
  1. Log in to the management domain vCenter Server at
    https://<vcenter_server_fqdn>/ui
    as
    administrator@vsphere.local
    .
  2. In the
    VMs and templates
    inventory, expand the management domain vCenter Server tree and expand the management domain data center.
  3. Right-click the primary NSX manager virtual machine and select
    Power
    Shut down Guest OS
    .
  4. In the confirmation dialog box, click
    Yes
    .
    This operation takes several minutes to complete.
  5. Repeat the steps for the remaining NSX Manager virtual machines.

Shut Down the VxRail Manager Virtual Machine in a VI Workload Domain with vSphere with Tanzu

Because the vCenter Server instance for the VI workload domain is already down, you shut down the VxRail Manager virtual machine from the ESXi host on which it is running.
  1. Using the VMware Host Client, log in as
    root
    to the ESXi host that runs the VxRail Manager virtual machine.
  2. In the navigation pane, click
    Virtual machines
    .
  3. Right-click the VxRail Manager virtual machine and select
    Guest OS
    Shut down
    .
  4. In the confirmation dialog box, click
    Yes
    .

Shut Down vSAN and the ESXi Hosts in a Virtual Infrastructure Workload Domain with vSphere with Tanzu

You shut down vSAN and the ESXi hosts in a VI workload domain with vSphere with Tanzu by preparing the vSAN cluster for shutdown, placing each ESXi host in maintenance mode to prevent any virtual machines being deployed to or starting up on the host, and shutting down the host.
In a VI workload domain with vSphere with Tanzu, the vCenter Server instance for the domain is already down. Hence, you perform the shutdown operation on the ESXi hosts by using the VMware Host Client.
  1. Turn on SSH on the ESXi hosts in the workload domain by using the SoS utility of the SDDC Manager appliance.
    1. Log in to the SDDC Manager appliance by using a Secure Shell (SSH) client as
      vcf
      .
    2. Switch to the
      root
      user by running the
      su
      command and entering the root password.
    3. Run this command.
      /opt/vmware/sddc-support/sos --enable-ssh-esxi --domain
      domain-name
  2. Log in to the first ESXi host in the workload domain cluster by using a Secure Shell (SSH) client as
    root
    .
  3. For a vSAN cluster, deactivate vSAN cluster member updates by running the command.
    esxcfg-advcfg -s 1 /VSAN/IgnoreClusterMemberListUpdates
    The command returns
    Value of IgnoreClusterMemberListUpdates is 1
  4. Repeat 2 and 3 on the remaining hosts in the cluster.
  5. On the first ESXi host per vSAN cluster, prepare the vSAN cluster for shutdown by running the command.
    python /usr/lib/vmware/vsan/bin/reboot_helper.py prepare
    The command returns
    Cluster preparation is done!
  6. Place the ESXi host in maintenance mode by running the command.
    esxcli system maintenanceMode set -e true -m noAction
    Ensure the prompt comes back after the command is complete.
  7. Verify that the host is in maintenance mode.
    esxcli system maintenanceMode get
  8. Repeat 6 and 7 on the remaining hosts in the workload domain cluster.
  9. Shut down the ESXi hosts in the workload domain cluster.
    1. Log in to the first ESXi host for the cluster at
      https://<
      esxi_host_fqdn
      >/ui
      as
      root
      .
    2. In the navigation pane, right-click
      Host
      and, from the drop-down menu, select
      Shut down
      .
    3. In the confirmation dialog box, click
      Shut down
      .
    4. Repeat the steps for the remaining hosts in the cluster.