Cloud Proxy Troubleshooting
Cloud proxy troubleshooting steps are provided to help you easily resolve issues that you may come across in
VMware Aria
Operations
. Before you proceed with troubleshooting, see the Cloud Proxy FAQ.
Installation and/or First Boot Failure
To verify the issue, check if/var/log/firstbootcontains a file named "Succeeded".
If not, the following problems could result in
VMware Aria
Operations
installation and/or first boot failure:- Unique Registration Key used while deployingCloud Proxyis invalid. To verify, check thecloud proxyconsole.Solution: Redeploycloud proxy.
Cloud Proxy VM is running, but the status is Offline in VMware Aria
Operations.
VMware Aria
Operations
.
To verify the connection, use the following commands: (For the complete list of commands, please see Using the Cloud Proxy Command-Line Interface.
# Overall status of cloud proxy:cprc-cli -s # Ping itself: ip addr ping <address> # Ping gateway: ip route ping <gateway> # Verify the connection outside the cloud proxy, ping 8.8.8.8 Note: If you are using a network proxy, use the /opt/vmware/share/vami/vami_config_net option#5 command to ensure you have the correct configuration for the testings.
The following problems could result in
VMware Aria
Operations
displaying the status of cloud
proxy
as offline. - Incorrect network proxy information in cloud proxy configuration.To verify the connection via a network proxy, use the following:curl -vvv --proxy http(s)://proxy_user:proxy_pass@proxy_ip:proxy_port -H 'Accept: application/json' -H 'Content-Type: application/json' -X GET https://<gateway url>/casa/security/ping (gateway url example - 10238.gw.dev.vrops-ops.com) To ignore SSL validation for a proxy server, use curl --proxy-insecure. With SSL validation the customer can provide Proxy Server certificate during cloud proxy deployment or re-configuration so that provided certificate from customer can be used to check the connection with curl with SSL certificate validation.Solution:
- SSH to theCloud ProxyVM and set theconnectretryto0in/storage/db/vmware-vrops-cprc/configuration/cprc.configurationto ensure that theCloud Proxyretries to connect.
- Shutdown theCloud ProxyVM.
- Update the network proxy configurations from thevCenter ServerVM options using the vApp options Edit OVF Details for a Virtual Machine.
- Boot theCloud ProxyVM.
- Required ports are not open.To verify:openssl s_client -showcerts -connect {address}:443 curl -v telnet://{address}:443 # Or, change the address to the machine you want to check: python -c "import socket; print(socket.socket(socket.AF_INET, socket.SOCK_STREAM).connect_ex(('127.0.0.1', 443)))" # If you get a !=0 response, the server is not listening to the port.Solution:
- SSH to theCloud ProxyVM and setconnectretryto0in/storage/db/vmware-vrops-cprc/configuration/cprc.configurationto ensure that theCloud Proxyretries to connect.
- Provide port access as mentioned in the prerequisite section of Configuring Cloud Proxies in VMware Aria Operations
- Boot theCloud ProxyVM.
- Invalid certificate.To verify:openssl s_client -showcerts -connect {address}:443Solution:
- SSH to theCloud ProxyVM and setconnectretryto0in/storage/db/vmware-vrops-cprc/configuration/cprc.configurationto ensure that theCloud Proxyretries to connect.
- Follow the steps mentioned in VMware KB Article, 83698.
- The logs folder/storage/logis running out of partition space.Solution: Remove log files to ensure that enough space is available. Note that this is an exceptional case. In normal conditions, log files are auto archived.
- One or more of the following services are down:httpd-north.service,haproxy.serviceandcollector.service.Solution:
- Check service status by running the following command:systemctl status <service name>.
- To start service, use the following command:systemctl start <service name>.
- Unique Registration Key expired.Solution: RedeployCloud Proxywith new Unique Registration Key.
Cloud proxy is online, and state of Cloud Account is Collecting, but status is Object Down.
Collecting
, but status is Object Down
. 
The following problem could result in
VMware Aria
Operations
displaying the state of Cloud Account as Collecting
, while the status is, Object Down
.- Incorrect account credentials.Solution: Check and update the credentials used while setting up the cloud account.
Cloud proxy status is stuck in Going Online.
Going Online
.
It can take up to 20 mins on first reboot, for the
cloud
proxy
to be registered and come online. Wait for the specified time to see if cloud proxy comes online. If it still does not come online, one or more of the following services are down: httpd-north.service
, haproxy.service
, and collector.service
.Solution:
- Check service status by running the following command:systemctl status <service name>
- To start service, use the following command:systemctl start <service name>.
Cloud proxy does not upgrade automatically, after the upgrade of VMware Aria
Operations
VMware Aria
Operations
There could be a few possible reasons why cloud proxy does not upgrade automatically after an upgrade of
VMware Aria
Operations
.- High network latency leading to PAK download failure. Latency of >500ms is not supported.Solution: See the VMWare KB article 80590 on how to manually upgrade cloud proxy via CLI.
- Upgrade status is stuck atRunningsince the previous upgrade had failed.Solution: Follow the steps given below to change the upgrade status.
- Stop the casa service:systemctl stop vmware-casa.service.
- Change the upgrade status fromRUNNINGtoNONEin the following files:./storage/db/vmware-vrops-cprc/status/cprc.upgrade.status ./storage/db/vmware-vrops-cprc/status/cprc.pak.status
- See the VMware KB article 80590 and run the manual upgrade.
Cloud proxy gets disconnected at regular intervals
There could be a few possible reasons why cloud proxy gets disconnected at regular intervals.
- Check the network connectivity and latency.
- Check if the cloud proxy VM can reach the DNS and use the NSlookup to validate the DNS connectivity.