[YSQL] [PG15 Online Upgrade] Capture pg_upgrade stdout in master logs #25900
Assignees
Labels
jira-originated
kind/enhancement
This is an enhancement of an existing feature
priority/low
Low priority
Comments
yugabyte-ci
added
jira-originated
kind/enhancement
hari90
added a commit
that referenced
this issue
Feb 7, 2025
… them in master logs Summary: - Redirect the stdout and stderr of the `pg_upgrade` and `initdb` process to the master logs. This will help in debugging errors from these processes. This uses the same logic currently in use by `ExternalMiniCluster` tests. - Improve the upgrade tests to check for exact errors strings in the logs. - Running the YB-specific checks in `pg_upgrade` at the start, as these catch issues like missing upgrade compatibility GUCs. This issue was found with the logging changes. - Use a separate pg data directory (`pg_upgrade_data`) for the upgrade. This helps avoid conflict with the `pg_data` folder used by the yb-tserver. All `pg_upgrade` logs and dump data will are placed in this directory, which also makes it simpler for post upgrade cleanup operations (#25907). Example output from yb-master logs ``` I0205 16:40:10.333971 1836642304 subprocess.cc:888] [pg_upgrade] Performing Consistency Checks I0205 16:40:10.334014 1836642304 subprocess.cc:888] [pg_upgrade] ----------------------------- I0205 16:40:10.439098 1836642304 subprocess.cc:888] [pg_upgrade] Checking expression pushdown is disabled ok I0205 16:40:10.447584 1836642304 subprocess.cc:888] [pg_upgrade] Checking attributes of the user used to access the old clusterChecking 'yugabyte' user attributes ok I0205 16:40:10.448410 1836642304 subprocess.cc:888] [pg_upgrade] Checking 'yugabyte' user attributes ok I0205 16:40:10.464574 1836642304 subprocess.cc:888] [pg_upgrade] Checking for all 3 system databases ok I0205 16:40:10.464596 1836642304 subprocess.cc:888] [pg_upgrade] ok I0205 16:40:11.043160 1836642304 subprocess.cc:888] [pg_upgrade] Checking database connection settings ok I0205 16:40:11.281297 1825746944 ysql_manager.cc:138] Checking if ysql major catalog upgrade is done I0205 16:40:11.286499 1836642304 subprocess.cc:888] [pg_upgrade] Checking for system-defined composite types in user tables ok I0205 16:40:11.473160 1836642304 subprocess.cc:888] [pg_upgrade] Checking for reg* data types in user tables ok I0205 16:40:11.473188 1836642304 subprocess.cc:888] [pg_upgrade] Checking for contrib/isn with bigint-passing mismatch ok I0205 16:40:11.678638 1836642304 subprocess.cc:888] [pg_upgrade] Checking for user-defined postfix operators ok I0205 16:40:11.963297 1836642304 subprocess.cc:888] [pg_upgrade] Checking for incompatible polymorphic functions ok I0205 16:40:12.090222 1836642304 subprocess.cc:888] [pg_upgrade] Checking for tables WITH OIDS ok I0205 16:40:12.281961 1825173504 ysql_manager.cc:138] Checking if ysql major catalog upgrade is done I0205 16:40:12.348526 1836642304 subprocess.cc:888] [pg_upgrade] Checking for invalid "sql_identifier" user columns ok I0205 16:40:12.472633 1836642304 subprocess.cc:888] [pg_upgrade] Creating dump of global objects ok I0205 16:40:13.284899 1825173504 ysql_manager.cc:138] Checking if ysql major catalog upgrade is done I0205 16:40:14.286425 1825173504 ysql_manager.cc:138] Checking if ysql major catalog upgrade is done I0205 16:40:15.288862 1834921984 ysql_manager.cc:138] Checking if ysql major catalog upgrade is done I0205 16:40:15.831990 1836642304 subprocess.cc:888] [pg_upgrade] Creating dump of database schemas ok I0205 16:40:15.884467 1826320384 master.cc:702] resp: entries { db_oid: 1 shm_index: 0 } I0205 16:40:15.903304 1834921984 master.cc:702] resp: entries { db_oid: 1 shm_index: 0 } I0205 16:40:16.010345 1840656384 master.cc:702] resp: entries { db_oid: 13260 shm_index: 3 } I0205 16:40:16.051797 1844670464 master.cc:702] resp: entries { db_oid: 1 shm_index: 0 } I0205 16:40:16.076397 1836642304 subprocess.cc:888] [pg_upgrade] Checking for prepared transactions ok I0205 16:40:16.076427 1836642304 subprocess.cc:888] [pg_upgrade] Checking for new cluster tablespace directories ok I0205 16:40:16.076438 1836642304 subprocess.cc:888] [pg_upgrade] I0205 16:40:16.076444 1836642304 subprocess.cc:888] [pg_upgrade] If pg_upgrade fails after this point, you must re-initdb the I0205 16:40:16.076449 1836642304 subprocess.cc:888] [pg_upgrade] new cluster before continuing. I0205 16:40:16.076454 1836642304 subprocess.cc:888] [pg_upgrade] I0205 16:40:16.076459 1836642304 subprocess.cc:888] [pg_upgrade] Performing Upgrade I0205 16:40:16.076464 1836642304 subprocess.cc:888] [pg_upgrade] ------------------ ``` #25900 Jira: DB-15212 Test Plan: Pg15UpgradeTest.UsersAndRoles YsqlMajorUpgradeDdlBlockingTest.CreateAndDropDBs Pg15UpgradeTest.CheckUpgradeCompatibilityGuc Reviewers: telgersma, asrivastava Reviewed By: asrivastava Subscribers: svc_phabricator, ybase, yql Differential Revision: https://phorge.dev.yugabyte.com/D41702
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-15212
The text was updated successfully, but these errors were encountered: