How to add a custom Instance Type and resize universes in YugabyteDB Anywhere using API – Yugabyte

Environment

YugabyteDB Anywhere (YBA) Version: 2.20.5 and later.
Provider Type: AWS Based Cloud Provider

Issue Description

When attempting to edit a universe on AWS to scale down or change an instance type (e.g., from c5.4xlarge to t3.2xlarge), administrators may face two problems:

The target instance type is not available in the dropdown list within the YBA UI.
After manually adding the instance type via the backend, the "Smart Resize" option is still not presented. The UI defaults to a "Full Move," which is a slower operation requiring downtime because it spins up new nodes and decommissions the old ones.

This occurs because "Smart Resize" is only permitted when the only change is to the instance type or an increase in volume size. If YBA detects any other change, such as a change in storage type (e.g., from GP2 to GP3) or the number of disks, it disables this option. This can happen if the underlying EBS volumes were manually modified on AWS, causing a metadata mismatch with what YBA has on record.

Overview

YBA does not automatically sync all available instance types from the cloud provider, popular types are hardcoded, and others must be added manually.

The resolution involves bypassing the UI limitations by using a series of API calls. First, you enable a feature flag to allow unsupported instances. Second, you manually add the desired instance type to the YBA metadata for the specific cloud provider. Finally, you use a specific resize_node API endpoint that executes a rolling upgrade.

Steps

The following steps detail how to perform a rolling instance type change when the standard UI method is not working.

1. Enable Unsupported Instances in YBA

This flag allows you to add instance types that are not hardcoded in the YBA platform.

Log in to the YBA UI with SuperAdmin credentials.
Navigate to Admin > Advanced > Global Configuration.
In the search bar, look for yb.internal.allow_unsupported_instances.
From the Action menu for this flag, set its value to true.

2. Verify Cloud Provider Configuration

A common reason for failure is a mismatch between the instance type's availability and the Availability Zones (AZs) configured in YBA.

First, confirm that your desired instance type is available in all the AZs that are configured for your universe's cloud provider in YBA. You can use the AWS CLI for this. For example:

# Replace values as per your environments

aws ec2 describe-instance-type-offerings --location-type availability-zone --filters="Name=instance-type,Values=" --region  --output table
aws ec2 describe-instance-type-offerings --location-type availability-zone --filters="Name=instance-type,Values=m5.2xlarge" --output table --no-cli-pager
-------------------------------------------------------------
|               DescribeInstanceTypeOfferings               |
+-----------------------------------------------------------+
||                  InstanceTypeOfferings                  ||
|+--------------+--------------------+---------------------+|
|| InstanceType |     Location       |    LocationType     ||
|+--------------+--------------------+---------------------+|
||  m5.2xlarge  |  us-west-2b        |  availability-zone  ||
||  m5.2xlarge  |  us-west-2-lax-1b  |  availability-zone  ||
||  m5.2xlarge  |  us-west-2d        |  availability-zone  ||
||  m5.2xlarge  |  us-west-2c        |  availability-zone  ||
||  m5.2xlarge  |  us-west-2-den-1a  |  availability-zone  ||
||  m5.2xlarge  |  us-west-2a        |  availability-zone  ||
||  m5.2xlarge  |  us-west-2-lax-1a  |  availability-zone  ||
|+--------------+--------------------+---------------------+|

In the YBA UI, check the configured AZs under Cloud Provider configuration.
If the command output shows that your target instance type is not available in one of the AZs configured in YBA, you must either:
- Recommended: Remove the unsupported AZ from the Cloud Provider configuration in the YBA UI. If the UI does not allow you to save the change, this may need to be done on the backend database with the help of Yugabyte support team. For example:
  
  NOTE: Please take a YBA database backup before moving further:
```
$cd /opt/yugabyte/software/<ybanywhere_version>/pgsql/bin
$./pg_dump \
-h localhost \
-U postgres yugaware | gzip -c >/tmp/platform_dump_$(date -u -I).sql.gz
```
```
yugaware=# delete from availability_zone where name = 'us-east-1a';
DELETE 1
yugaware=# delete from availability_zone where name = 'us-east-1f';
DELETE 1
yugaware=# delete from availability_zone where name = 'us-east-1c';
DELETE 1
```
Alternative: Request your AWS team to enable the instance type in the missing AZ.

3. Add the New Instance Type via API

This command directly inserts the new instance type's metadata into YBA.

IMPORTANT: The instanceTypeCode must be in lowercase (e.g., m5.2xlarge, not M5.2xlarge). Using incorrect casing will create a faulty entry that does not work and will need to be deleted from the backend.

Connect to the YBA node via SSH.

Execute the following curl command, replacing the placeholders with your specific values.

NOTE: This is sample command, actual payload can vary. You can refer this API doc for more details. https://api-docs.yugabyte.com/docs/yugabyte-platform/7c2d9bde8ca8d-create-an-instance-type

# Replace <YBA-IP>, <cUUID>, <Provider-UUID>, <API-Token>, and instance details

curl -ks --request POST --url https://<YBA-IP>/api/v1/customers/<cUUID>/providers/<Provider-UUID>/instance_types \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--header 'X-AUTH-YW-API-TOKEN: <API-Token>' \
--data '{
  "idKey": {
    "instanceTypeCode": "t3.2xlarge"
  },
  "active": true,
  "numCores": 8.0,
  "memSizeGB": 32.0,
  "instanceTypeDetails": {
    "volumeDetailsList": [
      {
        "volumeSizeGB": 250,
        "volumeType": "EBS",
        "mountPath": "/mnt/d0,/mnt/d1"
      }
    ],
    "tenancy": "Shared",
    "arch": "x86_64"
  }
}'

Find this in the YBA UI under Profile icon > User Profiles.
The UUID of your AWS cloud provider configuration.
Your personal API token from the YBA UI.

4. Trigger the Resize via API

Since the UI option is unavailable due to the metadata mismatch, this API call forces the resize operation in a rolling fashion, ensuring no downtime.

Execute the following API call from the YBA node, replacing the placeholders.

# Replace <YBA-IP>, <cUUID>, <uUUID>, <API-Token>, and the target instanceType

curl -ks --request POST --url https://<YBA-IP>/api/v1/customers/<cUUID>/universes/<uUUID>/upgrade/resize_node \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--header 'X-AUTH-YW-API-TOKEN: <API-Token>' \
--data '{
  "universeUUID": "<uUUID>",
  "forceResizeNode": false,
  "upgradeOption": "Rolling",
  "clusters": [
    {
      "clusterType": "PRIMARY",
      "userIntent": {
        "instanceType": "t3.2xlarge"
      }
    }
  ]
}'

The UUID of the universe you are resizing. This can be found in the URL when viewing the universe in the YBA UI.
"instanceType": The new, desired instance type you added in the previous step.

After running this command, you can monitor the progress of the resize task in the YBA UI under the Tasks tab.

Reference: SUPPORT-489

Related to