How to resolve YBA Consistency Check Error - "No rows updated performing consistency check, stale universe metadata"

Environment

YugabyteDB Anywhere: All version

Summary

YBA task fails with error

No rows updated performing consistency check, stale universe metadata. DB at version <N> set by YBA <HOST> (<UUID>) during task <TASK-UUID>. Task should be run from up to date YBA or contact Yugabyte Support to resolve.

This error typically appears in Yugabyte Backup and Restore operations managed via Yugabyte Anywhere (YBA), indicating a mismatch or staleness in the Universe metadata.

This error can be triggered in the following scenarios:

• Out-of-Band Operations: Operations were performed on the cluster directly (using CLI, scripts, etc.), bypassing YBA.
• High Availability (HA) and Failover: In HA setups, the error occurs if the YBA instance running the task is not fully up to date (such as a promoted standby with outdated metadata).
• Restored Backups/Manual Migration: A stale backup was restored, or migration to a new host led to metadata mismatches.

The core check is performed by comparing the sequence number (seq_num) tracked in the YBA application's metadata with the value in the yba_consistency_check table within the system_platform database. If the value in the database is higher than what YBA expects, the task is aborted to avoid further inconsistency.

• Disable Consistency Check:
  Go to YBA UI > Admin > Advanced > Temporarily disable the consistency check by setting the runtime flag yb.universe.consistency_check.enabled to false .  This will bypass the error and allow the operation to proceed.

  
  Warning: Only use this if you are certain there is no risk of concurrent or out-of-sync operations. In case of uncertainty, engage Yugabyte Support or Engineering.

   

• Synchronize Sequence Numbers:

  • Verify the values in the YBA UI and the system_platform.yba_consistency_check table from the DB nodes by connecting to ysqlsh.

    SELECT * FROM yba_consistency_check;

  • The table includes columns: seq_num, task_uuid, yw_uuid, and yw_host.
  • YBA expects the seq_num in its Universe metadata to match the value in the database for the corresponding universe.
• Manual Correction:
  • If you are certain there is no YBA HA or other active YBA servers, and no pending or stale tasks, you can update the seq_num in the database to match YBA's sequenceNumber (visible in exported Universe JSON or via edit_universe_details.py).

  • Example SQL to update:

    UPDATE yba_consistency_check SET seq_num = <expected_seq_num> WHERE <condition>;

  • Before making changes, always take a YBA database backup.
• HA and Migration Caution:
  • If this occurs in an HA setup or after migration, ensure that YBA Active and Standby are in sync before making manual changes.

If you are unsure about the state of YBA instances or pending tasks. If the error persists after following above steps. Contact Yugabyte Support to avoid data inconsistencies.

The yba_consistency_check table is a metadata tracking table located in the system_platform database. Its primary purpose is to help maintain consistency between the YBA application's view of the universe state and what is actually recorded in the backing database, especially in multi-node or high-availability (HA) environments.

Key columns in yba_consistency_check:

• seq_num: The current sequence number for the universe. This is incremented by YBA with each universe-changing operation.
• task_uuid: The task that last updated the sequence number.
• yw_uuid: The unique ID of the YBA instance that last updated the sequence.
• yw_host: The hostname of the YBA instance that last updated the sequence.

Table Example:

This table acts as a "single source of truth" for change ordering, helping to prevent out-of-order operations and ensuring only the most recent state-modifying task is recognized across all YBA nodes.

References: SUPPORT-691