Environment:

YugabyteDB Anywhere - all versions

Issue:

Disk Usage tile shows "No Data" in YugabyteDB Anywhere's Universe page. Refer the below screenshot.

Cause:

Below are two reasons for this issue.

1. The `node_exporter` service on the Yugabyte DB hosts is not running.

2. The `universe_details` is missing the "mount_root" section from the host details.

Resolution:

Follow the below steps to resolve the issue.

1. Make sure the `node_exporter` service is running on the DB hosts. If not please start/restart it.

$ ps -aux | grep node_exporter
prometh+ 681 0.0 0.7 727104 27868 ? Ssl Jul27 6:03 /opt/prometheus/node_exporter-1.5.0.linux-amd64/node_exporter --web.listen-address=:9300 --collector.textfile.directory=/tmp/yugabyte/metrics

To stop/start/restart use below command. 

systemctl restart node_exporter

2. If after the `node_exporter` restart the Disk Usage tiles remains same i.e with No Data then move to next step.

3. Form the YBA host run the GET request to see the universe details like below and look for `mount_roots` parameter. It will not be there in the faulty universe.

curl -k --request GET --url https://$YBA_HOST/api/v1/customers/$CUUID/universes/$UUUID   --header 'Content-Type: application/json' --header "X-AUTH-YW-API-TOKEN: $TOKEN" | jq '.universeDetails.nodeDetailsSet[].cloudInfo'

{
  "private_ip": "xxxxx",
  "public_ip": "xxxx",
  "instance_type": "t2.xlarge",
  "az": "yb-us-west-2a",
  "region": "us-west-2",
  "cloud": "onprem",
  "assignPublicIP": true,
  "useTimeSync": true,
}

However in a good universe the output will be like this:

{
  "private_ip": "xxxxx",
  "public_ip": "xxxx",
  "instance_type": "t2.xlarge",
  "az": "yb-us-west-2a",
  "region": "us-west-2",
  "cloud": "onprem",
  "assignPublicIP": true,
  "useTimeSync": true,
  "mount_roots": "/"
}

4. Edit the universe details using the article or below steps.

Take a Backup of the Yugabyte Anywhere Database:

For a YBA Installer or Yugabundle based install:

pg_dump -U postgres -d yugaware > platform_dump.sql

For Replicated based install:

sudo docker exec -it postgres pg_dump -U postgres -d yugaware > platform_dump.sql

For the Kubernetes based install:

kubectl exec -it <yugaware pod> -c postgres -n <namespace> -- pg_dump -U postgres yugaware > platform_dump.sql

Using the edit_universe_details.py Script:

Note: This process will open the universe details in a vi window where you can make necessary edits and save them with :wq

For a standalone install the script needs to be run from the YBA instance

On the YBA instance, navigate to the directory where the script is located:

cd /opt/yugabyte/devops/bin/

To execute the script, run the following on the YBA instance:

sudo ./edit_universe_details.py -i <universe_uuid> -t standalone

For a replicated install the script needs to be run from the YBA instance

To get the script, it can be copied from the yugaware container when on the YBA instance:

sudo docker cp yugaware:/opt/yugabyte/devops/bin/edit_universe_details.py .

To execute the script, run the following on the YBA instance:

sudo ./edit_universe_details.py -i <universe_uuid> -t docker

For a k8's install, the script needs to be run from outside of the pods

To get the script, it can be copied from the yugaware container in the yugaware platform pod:

kubectl cp <yugaware pod>:/opt/yugabyte/devops/bin/edit_universe_details.py ~/edit_universe_details.py -c yugaware -n <namespace>

To execute the script, run the following:

./edit_universe_details.py -i <universe_uuid> -t kubernetes -p <yugaware pod> -n <namespace>

Edits to make:

You need to add each section under the .cloudInfo with mount_roots like below (one section for example).

{
  "private_ip": "xxxxx",
  "public_ip": "xxxx",
  "instance_type": "t2.xlarge",
  "az": "yb-us-west-2a",
  "region": "us-west-2",
  "cloud": "onprem",
  "assignPublicIP": true,
  "useTimeSync": true,
  "mount_roots": "/"
}

NOTE: The value of `mount_roots` can differ per environment. If you are using custom data_dir for YBDB then put respective value.

Once the changes are committed using :wq!