Environment

• YugabyteDB Anywhere - 2.14

Issue

This issue was seen on an upgrade from v2.8.9 to v2.14.5, but is not necessarily limited to these versions.

Error reported in yugaware logs after the universe upgrade failed.

Unable to upgrade YSQL cluster: 
Failed to apply migration 'V19_6560_pg_collation_icu_70.sql' to a database postgres: 
Execute failed: 7, message: ERROR: duplicate key value violates unique constraint 
"pg_collation_name_enc_nsp_index" (pgsql error 23505) 
(aux msg ERROR: duplicate key value violates unique constraint 
"pg_collation_name_enc_nsp_index").

This issue caused by a race condition in the update and inserts in the pg_collation_icu_70 upgrade script, which is required to update the catalog.

Because it's a race condition, it does not always occur.

Resolution

Resolved by applying the upgrade_ysql script via yb-admin.
https://docs.yugabyte.com/preview/admin/yb-admin/#upgrade-ysql-system-catalog

Examples of running the command:

yb-admin --master_addresses <master addresses> --certs_dir_name <tls config dir> -timeout_ms 180000 upgrade_ysql

If successful, you should see the following output:

YSQL successfully upgraded to the latest version

• If --master_addresses option is not added, it will default to localhost:7100
• The default timeout is 60 seconds, but the upgrade can take longer so it is advisable to override this via the -timeout_ms option.
• Option --certs_dir_name is only required if TLS is configured.
  
  

Please note
As this is a race condition, the command above may fail and may need to be run multiple times.