Environment

• YugabyteDB Anywhere - 2.8.8 or earlier
• YugabyteDB Anywhere - 2.12.9 or earlier
• YugabyteDB Anywhere - 2.14.5 or earlier

Issue

The schedule_uuid column may not be correctly populated in backup records.

When YugabyteDB Anywhere (YBA) is processing backup expiry, it uses the schedule_uuid column to map backups to schedules in order to reprieve backups from scheduled deletion. If this column isn't populated, scheduled backups may be expired unexpectedly.

Resolution

Overview

This issue can be addressed by ensuring that the schedule_uuid column is correctly populated for all backups.

Workaround

In YBA 2.8 and 2.12 versions, only full universe backups are affected by this issue, so altering the backup configuration to select all the keyspaces individually instead of running a full universe backup will work around this issue. This workaround is not applicable to YBA 2.14.

Permanent Fix

A permanent fix for this issue is available starting in YBA versions 2.8.9, 2.12.10, and 2.14.6. This issue does not affect the 2.16 release. It is recommended to upgrade YBA to a version containing the fix.

Note: The permanent fix corrects the issue for new backups but will not correct any existing backups. Use the steps below to correct the backup records for existing backups.

In order to prevent existing backups from being deleted, the schedule_uuid column for these backups must be populated manually.

Steps

1. Connect to the YugabyteDB Anywhere Postgres.

For Replicated installs:

sudo docker exec -it postgres psql -U postgres yugaware

For Kubernetes installs:

kubectl exec <name of the pod> -n <namespace> -it -c postgres -- psql -U postgres yugaware

2. Run the following query to list the backups in the database:

SELECT backup_uuid, create_time, expiry, schedule_uuid,
backup_info::jsonb->'backupList'->0->>'scheduleUUID' AS backup_info_schedule_uuid
FROM backup
ORDER BY expiry DESC;

Confirm whether there are backup records for which the schedule_uuid column is not populated.

3. If there are affected backups, begin a transaction and populate the schedule_uuid column from the corresponding field in the backup_info JSON:

BEGIN;
UPDATE backup
SET schedule_uuid = (backup_info::jsonb->'backupList'->0->>'scheduleUUID')::uuid
WHERE schedule_uuid IS NULL
AND backup_info::jsonb->'backupList'->0->>'scheduleUUID' IS NOT NULL;

4. Review the changes made to the backup records (the UPDATE query should have updated the schedule_uuid column to match the backup_info_schedule_uuid column): 

SELECT backup_uuid, create_time, expiry, schedule_uuid,
backup_info::jsonb->'backupList'->0->>'scheduleUUID' AS backup_info_schedule_uuid
FROM backup
ORDER BY expiry DESC;

If everything looks good, COMMIT; the transaction. Otherwise, revert the changes by running ROLLBACK;.