Environment

• Yugabyte Platform - All Versions

Issue

From Yugabyte Anywhere UI the Universe creation or upgrade fails with below error message which can be seen in application.log file inside the yugaware container or under logs section in Yugabyte Anywhere UI. 

ERROR! Problem running vault password script /opt/yugabyte/yugaware/data/keys/21a27268-3875-451d-930a-4b373a3e0370/yugapoc-v6-key.vault_password ([Errno 8] Exec format error: '/opt/yugabyte/yugaware/data/keys/21a27268-3875-451d-930a-4b373a3e0370/yugapoc-v6-key.vault_password'). If this is not a script, remove the executable bit from the file.

Cause

The universe creation or upgrade scripts checks the permission of vault_passwordfile and when it finds the file has executable permission it marks this as a security issue and fails. Since this is a private key file and should have only read permissions for other users. For example, in an idle cluster on the platform node, the permission will look like below with executable bit enabled:

/opt/yugabyte/yugaware/data/keys/42e70c27-4349-4fa3-9c73-3f33ac0a641e:
-rw-r--r--. 1 root root   32 Nov 10 15:10 yb-dev-aws-cloud-provider-for-support_42e70c27-4349-4fa3-9c73-3f33ac0a641e-key.vault_password

Resolution

Steps

1. SSH into the platform node.

2. Go to the data directory associated with the universe being created/upgraded:

cd /opt/yugabyte/yugaware/data/keys/<universe ID>

Where: <universe ID> is the UUID of the universe being created or upgraded.

Tip: Press the TAB key while listing the dir after /keys and you will see all universe dirs.

3. Change the permission of the vault_passwordfile to read (and write for root) only. For example:

sudo chmod 644 /opt/yugabyte/yugaware/data/keys/<universe ID>/fileprefix-key.vault_password

4. Exit the SSH session to the platform node. 

5. Navigate to the Yugabyte Anywhere UI and retry the Universe creation or upgrade.