Tools like log-dump, rocksdb_dump and sst_dump are useful to dump the physical contents of a file without having a running cluster. This is useful for offline investigations. Their major drawback that they do not fully show the user facing data which is a combination of multiple SSTs and rocksDBs (RegularDB and IntentsDB), and also they do not work if the files are encrypted.
dump_tablet_data
For online debugging, and to get the user visible view of the tablet in a human readable format, dump_tablet_data command of yb-ts-cli can be used.
yb-ts-cli dump_tablet_data <tablet_id> (<dest_path> | HASH_ONLY) [read_ht]
This command performs a Intent aware iteration (the same scan code used by YCQL and YSQL) to get the user visible data from the tablet.
- It works for both YSQL and YCQL tablets.
- It works on both the leader and follower peers (so can be used to validate the contents of the followers with the leader)
- It takes in an optional
read_htso that we can read the data across peers at the same time.
NOTE: In order to use on large tablets without hitting read restart errors, user would have to increase timestamp_history_retention_interval_sec or setup Snapshot Schedules.
Ex:
Sample output:
Tablet ID: 09bc4add191a47ff9dd858f84510195d
Peer UUID: fa8dc06a7dbc4be39078b4bd8495baf9
Peer Role: LEADER
Read HT: { physical: 1768350064290874 }
Table Info:
<TableInfoPB here>
Rows:
5, test5
1, test1
6, test6
7, test7
9, test9
Row count: 5
XOR hash: 17016537574803746Minimum available versions:
2025.2.1, 2024.2.8, 2025.1.4
XOR Hash
The tool can also skip dumping the entire contents of the tablet (which can be several GB in size) and only return the XOR hash and row counts (HASH_ONLY).
XOR is commutative, which makes it possible to combine with multiple tablets. This enables validation of scenarios like tablet split, verify the data in the parent and children tablets should be the same, so the XOR hash of parent should be the ^ of its child tablet XOR hashes.
NOTE: To easily get the table level XOR hash from the leader tablet peers use yb_admin get_table_hash command.
Comments
0 comments
Please sign in to leave a comment.