Environment:

YugabyteDB Anywhere - all versions

Issue:

Disk Usage tile shows "No Data" in YugabyteDB Anywhere's Universe page. Refer the below screenshot.

Cause: 

Below are two reasons for this issue. 

1. The `node_exporter`  service on the Yugabyte DB hosts is not running. 

2. The `universe_details` is missing the "mount_root" section from the host details. 

Resolution:

Follow the below steps to resolve the issue. 

1. Make sure the `node_exporter` service is running on the DB hosts. If not please start/restart it. 

$ ps -aux | grep node_exporter
prometh+ 681 0.0 0.7 727104 27868 ? Ssl Jul27 6:03 /opt/prometheus/node_exporter-1.5.0.linux-amd64/node_exporter --web.listen-address=:9300 --collector.textfile.directory=/tmp/yugabyte/metrics

To stop/start/restart use below command. 

systemctl restart node_exporter

2. If after the `node_exporter` restart the Disk Usage tiles remains same i.e with No Data then move to next step. 

3. Form the YBA host run the GET request to see the universe details like below and look for `mount_roots` parameter. It will not be there in the faulty universe. 

curl -k --request GET --url https://$YBA_HOST/api/v1/customers/$CUUID/universes/$UUUID   --header 'Content-Type: application/json' --header "X-AUTH-YW-API-TOKEN: $TOKEN" | jq '.universeDetails.nodeDetailsSet[].cloudInfo'

{
  "private_ip": "xxxxx",
  "public_ip": "xxxx",
  "instance_type": "t2.xlarge",
  "az": "yb-us-west-2a",
  "region": "us-west-2",
  "cloud": "onprem",
  "assignPublicIP": true,
  "useTimeSync": true,
}

However in a good universe the output will be like this: 

{
  "private_ip": "xxxxx",
  "public_ip": "xxxx",
  "instance_type": "t2.xlarge",
  "az": "yb-us-west-2a",
  "region": "us-west-2",
  "cloud": "onprem",
  "assignPublicIP": true,
  "useTimeSync": true,
  "mount_roots": "/"
}

4. Edit the universe details using the article or below steps. 

NOTE: Please take a backup of Yugabyte Anywhere database first before making any changes. Below are the steps to take database backups.

* For Replicated based install:

sudo docker exec -it postgres pg_dump -U postgres -d yugaware > platform_dump$(date -u -I).sql

* For the K8 Based install:

kubectl exec pod/platform-yugaware-0 -n yugaware -it -c postgres -- pg_dump -U postgres yugaware > platform_dump$(date -u -I).sql

The easiest way is to do via edit_universe_details.py available on YBA host. Here are details steps.

• SSH into the YBA host and copy the script on node from the yugaware container like this.

ubuntu@ybahost:/opt/downloads$ sudo docker cp yugaware:/opt/yugabyte/devops/bin/edit_universe_details.py /home/ubuntu/
ubuntu@ybahost:/opt/downloads$ ll /home/ubuntu/
-rwxrwxr-x 1 root   root   5337 May 27 00:16 edit_universe_details.py* 

• Then run the script like below.

$ /home/ubuntu/edit_universe_details.py -i <Universe UUID> -t docker

• • The script will automatically consult Postgres and open the file for editing using vi. Go into insert mode in vi editor by pressing I.
• You need to add each section under the .cloudInfo with mount_roots like below (one section for example). 

{
  "private_ip": "xxxxx",
  "public_ip": "xxxx",
  "instance_type": "t2.xlarge",
  "az": "yb-us-west-2a",
  "region": "us-west-2",
  "cloud": "onprem",
  "assignPublicIP": true,
  "useTimeSync": true,
  "mount_roots": "/"
}

NOTE: The value of `mount_roots` can differ per environment. If you are using custom data_dir for YBDB then put respective value. 

• Once the changes are committed using :wq!