Environment

• YBDB Upgrade to 2024.2 and above
• YugabyteDB Anywhere (YBA) running xCluster/Change Data Capture (CDC)

Issue

Upon upgrading to YugabyteDB v2024.2 or later, the upgrade fails, and the yb-tserver service repeatedly crashes with the following symptoms:

[yugabyte@yugabyte-test~]$ systemctl --user status yb-tserver
● yb-tserver.service - Yugabyte tserver service
Loaded: loaded (/opt/yugabyte/.config/systemd/user/yb-tserver.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: core-dump) since Wed 2025-04-02 14:43:51 MST; 1s ago
Process: 2116853 ExecStart=/opt/yugabyte/tserver/bin/yb-tserver --flagfile /opt/yugabyte/tserver/conf/server.conf (code=dumped, signal=ABRT)
Process: 2116834 ExecStartPre=/opt/yugabyte/bin/clock-sync.sh (code=exited, status=0/SUCCESS)
Main PID: 2116853 (code=dumped, signal=ABRT)

The journalctl logs display the following lines:

Mar 27 16:40:38 yugabytem01-3 yb-master[1712934]: E0327 16:40:38.812829 1712934 log.cc:261] Invalid value '14400' for flag 'cdc_wal_retention_time_secs': Must be greater than cdc_intent_retention_ms (in seconds)
Mar 27 16:40:38 yugabytem01-3 yb-master[1712934]: F0327 16:40:38.813004 1712934 flags.cc:681] Check failed: _s.ok() Bad status: Invalid argument (yb/util/flags/flags.cc:864): ERROR: failed validation of new value '14400' for flag 'cdc_wal_retention_time_secs'
Mar 27 16:40:38 yugabytem01-3 yb-master[1712934]: : log.cc:261] Invalid value '14400' for flag 'cdc_wal_retention_time_secs': Must be greater than cdc_intent_retention_ms (in seconds)

Note: The yb-master service may also fail with the same error.

Root Cause

Starting with version v2024.2, the default values for certain CDC retention flags have changed:

If cdc_wal_retention_time_secs is explicitly set to a value less than cdc_intent_retention_ms (in seconds), the server fails to start, producing logs such as:

Invalid value '14400' for flag 'cdc_wal_retention_time_secs': Must be greater than cdc_intent_retention_ms (in seconds)
Check failed: _s.ok() Bad status: ERROR: failed validation of new value '14400' for flag 'cdc_wal_retention_time_secs'

Resolution

1. Rollback the Upgrade:

   • Use the rollback feature in YBA to revert the database to the previous version.

2. Adjust the cdc_wal_retention_time_secs Flag:

   • Remove the explicit setting of the cdc_wal_retention_time_secs flag from your configuration.

3. Retry the Upgrade:

   • After making the above adjustments, retry the failed upgrade task from the task page.

Note: This can occur in various YugabyteDB Anywhere tasks, such as updating G-Flags, so this article applies to those cases too. 

By following these steps, you can resolve the startup issue and successfully upgrade to YugabyteDB v2024.2 or later.