[DocDB] PgLibPqTest.MasterRestartReadPastGlobalLimit fails in TSAN #25136
Assignees
Labels
area/docdb
YugabyteDB core features
kind/enhancement
This is an enhancement of an existing feature
kind/failing-test
Tests and testing infra
priority/medium
Medium priority issue
Comments
spolitov
added
area/docdb
yugabyte-ci
added
kind/failing-test
spolitov
added a commit
that referenced
this issue
Dec 1, 2024
…artReadPastGlobalLimit TSAN mode Summary: The test performs 10k iterations, in TSAN mode it takes 0.1s to perform single iteration in TSAN mode. So test just times out because 15 minutes is required to complete such number of iterations. Decreased number of iterations to 1k for sanitizers. Jira: DB-14286 Test Plan: ./yb_build.sh tsan --gtest_filter PgLibPqTest.MasterRestartReadPastGlobalLimit -n 40 Reviewers: jason, patnaik.balivada Reviewed By: patnaik.balivada Subscribers: ybase, yql Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D40381
vaibhav-yb
pushed a commit
to vaibhav-yb/yugabyte-db
that referenced
this issue
Dec 10, 2024
…sterRestartReadPastGlobalLimit TSAN mode Summary: The test performs 10k iterations, in TSAN mode it takes 0.1s to perform single iteration in TSAN mode. So test just times out because 15 minutes is required to complete such number of iterations. Decreased number of iterations to 1k for sanitizers. Jira: DB-14286 Test Plan: ./yb_build.sh tsan --gtest_filter PgLibPqTest.MasterRestartReadPastGlobalLimit -n 40 Reviewers: jason, patnaik.balivada Reviewed By: patnaik.balivada Subscribers: ybase, yql Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D40381
ddhodge
added a commit
that referenced
this issue
Dec 12, 2024
…#25112) * changes to add transaction ordering deprecation * doc update for CDC * add missing content * added note * addressed review comments * addressed review comments * changed note type to warning * made edits and removed 2024.2 changes * reverted change for 2024.2 * Apply suggestions from code review Co-authored-by: Dwight Hodge <[email protected]> * [PLAT-16160]Add runtime conf flag for Off-cluster PITR AcceptedPublic Summary: Converted the feature flag to runtime flag. Show New scheduled Policy UI when 'yb.ui.feature_flags.off_cluster_pitr_enabled' is enabled and hide the old one. Show PITR column in backup list when 'yb.ui.feature_flags.off_cluster_pitr_enabled' is enabled and hide the old one. Test Plan: Tested manually Reviewers: lsangappa Reviewed By: lsangappa Subscribers: yugaware Differential Revision: https://phorge.dev.yugabyte.com/D40136 * [#24876] CDC: Cleanup metrics for deleted state table entries Summary: Currently we do not remove metrics corresponding to the deleted state table entries, for example, on deleting a CDC stream. These metrics hang around until there's a tserver restart. This might cause any external applications to read false values for metric objects which do not exist. This diff introduces code changes which address the above issue by deleting the metric entity when the corresponding CDC stream is deleted. Jira: DB-13995 Test Plan: The following tests have been introduced: ``` ./yb_build.sh --cxx-test cdcsdk_ysql-test --gtest_filter CDCSDKYsqlTest.TestCDCMetricRemovalUponStreamDeletion ./yb_build.sh --cxx-test cdcsdk_ysql-test --gtest_filter CDCSDKYsqlTest.TestCDCMetricRemovalUponSlotDeletionForLogicalReplication ./yb_build.sh --cxx-test cdcsdk_ysql-test --gtest_filter CDCSDKYsqlTest.TestMetricObjectRemovalAfterStreamExpiration ``` Reviewers: skumar, sumukh.phalgaonkar, siddharth.shah Reviewed By: sumukh.phalgaonkar Subscribers: ycdcxcluster Differential Revision: https://phorge.dev.yugabyte.com/D40055 * [#25057] YSQL,QueryDiagnostics: Fix testYbQueryDiagnosticsStatus Summary: testYbQueryDiagnosticsStatus has been failing due to a bug within pg_mkdir_p function. Current diff handles the bug and formats the test. Jira: DB-14188 Test Plan: ./yb_build.sh --java-tests 'org.yb.pgsql.TestYbQueryDiagnostics#testYbQueryDiagnosticsStatus' Reviewers: asaha Reviewed By: asaha Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D40209 * [#25111] CDC: Fix cleanup tests flakiness Summary: The test utility method CDCSDKYsqlTest::CheckTabletsInCDCStateTable() tries to get all the state table entries with non-max checkpoint and compare their equality with the expected set of entries. However when this method tries to filter out max checkpoint entries it incorrectly handles the optional checkpoint field in state table row entry. Because of this the filtering logic does not work and leads to test flakiness. This diff fixes the incorrect handling of the optional field. Jira: DB-14258 Test Plan: Jenkins: test regex: .*CDC.* Reviewers: skumar, siddharth.shah Reviewed By: siddharth.shah Subscribers: ycdcxcluster Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D40311 * Fix swaggerGen on master Summary: UTs are broken on master due to missing swagger Gen run. Test Plan: N/A Reviewers: skurapati Reviewed By: skurapati Subscribers: yugaware Differential Revision: https://phorge.dev.yugabyte.com/D40326 * [#24886] YSQL: Fix major catalog upgrade for colocated tables Summary: Upgrade of colocated tables was failing because 1) The OID for the tablegroup for the table wasn't being properly restored. 2) YB properties including the colocation ID weren't being dumped by ysql_dump. Fix both issues, and add a test. Jira: DB-14006 Test Plan: Jenkins ./yb_build.sh release --cxx-test pg15_upgrade-test --gtest_filter Pg15UpgradeTest.ColocatedTables --with_tests Reviewers: telgersma, fizaa Reviewed By: telgersma Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D40260 * [#25075] YSQL: ON CONFLICT fails to properly update index Summary: We figured out that INSERT .. ON CONFLICT UPDATE failed to properly update a secondary index. It looks like the problem is because of index update optimization, which is enabled by default. To mitigate the problem we disable the optimization by default. Revert "[#18822] YSQL: Enable optimization to skip redundant update operations in 2024.2 by default" This reverts commit 92e026f33cd3215d348449d070b1c90f0916d063. Jira: DB-14205 Test Plan: Jenkins: urgent Reviewers: kramanathan, smishra Reviewed By: smishra Subscribers: ybase, yql Differential Revision: https://phorge.dev.yugabyte.com/D40300 * [PLAT-13992] Remove forbidden to override flags (GflagsUtil.java) from gflags UI Summary: Removing gflags that are forbidden to override from UI (The request will fail anyway if the user attempts to set them) Test Plan: check if UI shows flags Reviewers: vbansal Reviewed By: vbansal Subscribers: yugaware Differential Revision: https://phorge.dev.yugabyte.com/D40252 * [#25043] YSQL: Fix flakey YbGinIndex upgrade test Summary: https://github.com/yugabyte/yugabyte-db/issues/20959 describes what is basically an expected race condition for DDLs. The test consistently fails if I increase the number of times we drop / create the indexes: ```lang=diff --- src/yb/integration-tests/upgrade-tests/pg15_upgrade-test.cc +++ src/yb/integration-tests/upgrade-tests/pg15_upgrade-test.cc @@ -1296,12 +1296,14 @@ TEST_F(Pg15UpgradeTest, YbGinIndex) { auto conn = ASSERT_RESULT(cluster_->ConnectToDB()); ASSERT_NO_FATALS(check_and_insert_rows(conn, num_rows)); - // test dropping and recreating the indexes, to validate that these DDLs - // still work as expected in pg15 - for (const auto &index : kGinIndexes) { - ASSERT_OK(conn.ExecuteFormat("DROP INDEX $0", index.first)); + for (int i = 0; i < 10; i++) { + // test dropping and recreating the indexes, to validate that these DDLs + // still work as expected in pg15 + for (const auto &index : kGinIndexes) { + ASSERT_OK(conn.ExecuteFormat("DROP INDEX $0", index.first)); + } + ASSERT_NO_FATALS(create_indexes_with_retry()); } - ASSERT_NO_FATALS(create_indexes_with_retry()); ASSERT_NO_FATALS(check_and_insert_rows(conn, num_rows)); } ``` Randomly, when creating an index, the test would fail with the error ``` [Not found (yb/master/catalog_manager.cc:11601): LookupByIdRpc(tablet: a70099ab24ae47c79b8b41f879536f01, num_attempts: 1) failed: Tablet deleted: Table deleted at 2024-11-25 09:17:36 EST (split child tablet IDs [])] (pgsql error XX000) (aux msg ERROR: ERROR: [Not found (yb/master/catalog_manager.cc:11601): LookupByIdRpc(tablet: a70099ab24ae47c79b8b41f879536f01, num_attempts: 1) failed: Tablet deleted: Table deleted at 2024-11-25 09:17:36 EST (split child tablet IDs [])]) ``` Retrying the `CREATE INDEX` statements when they fail avoids this error. Jira: DB-14177 Test Plan: On mac and my devserver: ``` ./yb_build.sh release --cxx-test pg15_upgrade-test --gtest_filter Pg15UpgradeTest.YbGinIndex -n 60 ``` Jenkins: test regex .*YbGinIndex.* Reviewers: fizaa Reviewed By: fizaa Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D40255 * [#25000] xCluster: Fix xcluster_upgrade-test Summary: When AutoFlags are promoted or demoted, the tservers are notified only on the next heartbeats. The xCluster tets wait for xCluster to receive and process these AutoFlags configs, but do not wait for the source tservers to even receive them. This fix addresses this issue by reducing the heartbeat time, and forcing a sleep after every AutoFlags change. Fixes #25000 Jira: DB-14144 Test Plan: xcluster_upgrade-test Reviewers: jhe, slingam, xCluster Reviewed By: jhe Subscribers: ybase Differential Revision: https://phorge.dev.yugabyte.com/D40189 * [PLAT-16198] Ansible error parsing in shell process handler does not work Summary: It was parsing the wrong input to find the ansible task error. Ansible task errors are in stderr. Python error is in stdout. Test Plan: Manually tested by failing the ansible tasks. Screenshots attached. Also tested by copying the tmp output files locally. Before this fix: {F313328} With this fix: {F313329} Reviewers: amalyshev, cwang, nbhatia, muthu, svarshney Reviewed By: svarshney Subscribers: yugaware Differential Revision: https://phorge.dev.yugabyte.com/D40310 * [#24829] CDC: Fix GetChanges WAL segment reading and computation of safe hybrid time for GetChanges resposne Summary: During a CDC QA run, we encountered a data loss scenario caused by an interaction between transaction commit timing and WAL segment rollover. WAL segment 'X' was active when transaction 'T' was still a running transaction. Due to concurrent segment rollover, the transaction's APPLY operation was appended in the subsequent segment 'X+1'. The following sequence of events led to data loss: 1. Initial GetChanges Request: - Requested opid index present in segment 'X' - No CDC related WAL OPs were found in segment 'X' between [requested op id index, last index in 'X'] - `committed_op_id_index` (pointed to last index of segment 'X') lagged behind `majority_replicated_op_id_index`, hence we did not scan segment 'X+1'. - Finally, we returned empty response with incorrect safe hybrid time (=tablet leader safe time) 2. Subsequent GetChanges Request: - Successfully read segment 'X+1' ( as committed_op_id_index had advanced to segment 'X+1') - UPDATE_TXN_OP of transaction 'T' was read but CDC filtered it out as we do not ship WAL OPs having commit_time < previously returned safe hybrid time - Resulted in transaction 'T' never being shipped. Root Cause: We violated a critical invariant: the tablet leader safe time should only be returned in GetChanges responses after reading the entire WAL. But in the 1st Getchanges call, we returned the tablet leader safe time having only read till segment 'X', leading to improper transaction filtering and subsequent data loss. This diff makes the following changes to fix the above mentioned data loss issue: 1. When reading the WAL segment by segment, we will now wait for the `committed_op_id_index` to be >= last index in the segment. If the deadline is reached while waiting, we will return 0 WAL ops and indicate to wait for WAL update. 2. Identification of active segment in GetConsistentWALRecords has been changed. Earlier, we were inferring active segment by the value of `consistent_stream_safe_time_footer`. If it was kInvalid, then we concluded its an active segment. But because of changes in 1), that conclusion no longer holds true. If we reach the deadline while waiting for `committed_op_id_index` to be >= last index in segment, then the value of `consistent_stream_safe_time_footer` will still be invalid. Hence, we are now using a dedicated variable (`read_entire_wal`) that will notify if we actually read the active segment in `ReadReplicatedMessagesInSegmentForCDC()`. 3. Value of `safe_hybrid_time` returned in GetChanges response. We have the following scenarios which will decide the safe_hybrid_time sent in the response: - **Case-1**: Read the active segment -> return the last record's commit_time if valid records are found. Else, return tablet leader safe time (existing logic). - **Case-2**: Deadline reached while waiting for `committed_op_id_index` to be >= last index in the WAL segment. There are 2 ways in which we can reach deadline. For both these ways, we want to wait for WAL to be updated. - Deadline reached after reading multiple segments -> return request safe hybrid time. - Deadline reached on the 1st segment read in GetChanges call -> return request safe hybrid time. - **Case-3:** requested index (from_op_id_index) >= `committed_op_id_index` - If all RAFT replicated Ops are not yet APPLIED (i.e. `committed_op_id_index` != `majority_replicated_op_id_index`), then we indicate to wait for WAL update and return request safe hybrid time. - If all RAFT replicated OPs are also APPLIED (i.e. `committed_op_id_index` == `majority_replicated_op_id_index`), then we have following scenarios: - If the condition in case-3 was hit after reading multiple segments, then return footer value (`min_start_time_running_txns`) of last read segment. - If the condition in case-3 was hit on the 1st segment read in GetChanges call, then return tablet leader safe time as this particular case implies we have read the entire WAL. Additionally, we have added a WARNING log when the invariant for response safe time is broken. Jira: DB-13935 Test Plan: Jenkins Reviewers: skumar, sumukh.phalgaonkar Reviewed By: skumar Subscribers: ybase, ycdcxcluster Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D39925 * [PLAT-16194] Adding preflight check for python3.10/3.11 Summary: With recent changes to upgrade Ansible D32451 we now require python 3.10 or 3.11 on the node to run YBA. This diff updates the python check to only accept 3.10 or 3.11. We also can move the openSSL check to its own check as we don't need to conditionally perform it based on the python version, it is part of the base requirements (as it was only relevant for python 3.10+ anyways). Test Plan: run check on node without python3.10 or 3.11, install python3.11 and ensure check passes Reviewers: dshubin, sanketh Reviewed By: dshubin Subscribers: yugaware Differential Revision: https://phorge.dev.yugabyte.com/D40298 * [#25107] xCluster: RemoveNamespaceWhenSourceIsDown: Skip health checks when source is down Summary: When source is down the health check will fail, so disable it during validation. Fix DropAllTables to wait for safe time to be healthy before checking row counts. Fixes #25107 Jira: DB-14253 Test Plan: XClusterDBScopedTest.RemoveNamespaceWhenSourceIsDown XClusterDBScopedTest.DropAllTables Reviewers: jhe, xCluster Reviewed By: jhe Subscribers: ybase Differential Revision: https://phorge.dev.yugabyte.com/D40306 * [#25103] DocDB: Handle multiple splits, PITR restores and drop table correctly Summary: This change fixes one scenario that https://github.com/yugabyte/yugabyte-db/commit/79a69958177d did not handle completely resulting in a FATAL after multiple splits and PITR restores in the following sequence: # t0 - Only tablet T11 exists. # t1 - T11 splits into T12, T13. # t2 - PITR restore is performed to t0 which brings back T11 and hides T12, T13 with hide time - t2. # t3 - T11 again splits into T14, T15 at the same partition key boundaries as T12,T13. T11 is marked as hidden with hide time - t3 # t4 - Table gets dropped hiding - T14, T15 with hide time - t4. # t5 - PITR is performed to t4+ which reloads the syscatalog. # At this point the following tablets exist in hidden state with different hide times: T11(t3), [T12(t2),T13(t2)], [T14(t4),T15(t4)]. Attempts to load [T12(t2),T13(t2)] and [T14(t4),T15(t4)] which have the same key ranges and split depth results in the following fatal: ``` Failed to load sys catalog: Illegal state (yb/master/catalog_entity_info.cc:654): Failed while visiting tablets in sys catalog: System catalog snapshot is corrupted or built using different build type: TabletInfo object freed during load: Two tablets with the same partition key start and split depth: committed_consensus_state ``` Added a unit-test for this and it fails without the fix. **Fix: ** The fix is to avoid adding any tablets that were marked hidden even before the table is dropped to the partitions_ structure. This ensures that on a drop table, only the set of tablets that were active are added to the partitions_ structure. Jira: DB-14249 Test Plan: ybt yb-admin-snapshot-schedule-test YbAdminSnapshotScheduleTest.ReadsWithMultipleSplitsPitrsAndDrop Reviewers: zdrudi, asrivastava Reviewed By: zdrudi Subscribers: tfoucher, ybase Differential Revision: https://phorge.dev.yugabyte.com/D40314 * [#25072] DocDB: Fix false corruption detection during colocated index read Summary: When index is colocated to the indexed table we perform index read using single query. At the first step the index table is read, to fetch row keys from the indexed table. On the second step indexed table is read using those keys. If row key does not exist on the second step, the corruption is being reported. It could happen that transaction apply happens in parallel to read and this row was just removed by recently committed transaction. So read finds this row on the first step, but does not see it on the second step raising the corruption error. Actually read restart mechanism detects that read is not consistent and should be restarted. But this particular corruption detection logic ignores read restart. The failure message looks like this one: ``` ERROR: ybctid DocKey([], [16754]) not found in indexed table. index table id is 00004007000030008000000000000a76 ``` Fixed by checking read restart before reporting such corruption. Updated param naming in PgDdlAtomicityStressTest to be more used friendly. Also fixed the following issues found by this test: 1) Potential crash because of head use after free if AbortTransaction response is received during tablet shutdown. 2) TServer could wait until OpenTable timeout (2min) during shutdown. 3) TServer could wait tablet validation timeout (2min) during shutdown. Jira: DB-14202 Test Plan: ./yb_build.sh fastdebug --gcc11 --gtest_filter PgDdlAtomicityStressTest.Main/colocated -n 20 Reviewers: myang, dmitry Reviewed By: myang Subscribers: dmitry, yql, ybase Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D40235 * [PLAT-16203]Skip running prechecks if Node2Node certs have expired Summary: We should skip running prechecks if certificates have expired. This will otherwise block rotate certs tasks on a univers e Test Plan: Tested locally by rotating certificates on a universe with expired certs Reviewers: svarshney, nsingh, nbhatia Reviewed By: svarshney, nbhatia Subscribers: yugaware Differential Revision: https://phorge.dev.yugabyte.com/D40331 * [PLAT-16202] Use the tmp_directory configured in ynp config file Summary: Use the tmp_directory configured in ynp config file Test Plan: Manually tested script files are generated at the specified directory Reviewers: nbhatia, skhilar Reviewed By: nbhatia Subscribers: yugaware Differential Revision: https://phorge.dev.yugabyte.com/D40324 * [#23331] yugabyted: Add pg_isready check to start and status command. Summary: * Added the check pg_isready for each node during start and status command. * Added the field `YSQL Status` in the status string displayed during start and status command. * During the restart of a cluster if a node is in Bootstrapping step, yugabyted status will return the same status for `YSQL Status` * Added a new `Output.ANIMATION_STOP` status for Output.update_animation() function. * Giving `Output.ANIMATION_STOP` and empty msg string to Output.update_animation() will remove the spinner that was started with Output.init_animation(). Jira: DB-12256 Test Plan: Manual Testing Reviewers: nikhil Reviewed By: nikhil Subscribers: sgarg-yb Differential Revision: https://phorge.dev.yugabyte.com/D34597 * [PLAT-13112]Block deletion of in use-releases Summary: Adding finalizers in ReleaseReconciller.onAdd() to block deletion of in-use releases. Removing finalizers in ReleaseReconciller.onDelete() and after ybuniverse gets deleted. Test Plan: Tested following scenarios - - Created a release -> check finalizers present in cr -> Delete release - Created a release -> Create universe -> delete release (blocked) -> Delete universe -> Release gets GC'ed - Created a release -> Create universe -> Delete universe -> Release stays active Reviewers: anijhawan, vkumar Reviewed By: vkumar Subscribers: yugaware Differential Revision: https://phorge.dev.yugabyte.com/D37451 * [#24293] YSQL, QueryDiagnostics: Fix PgssReset test Summary: D39523 revision enabled ASH by default, which started raising extra warning messages for `testPgssResetBetweenDiagnostics` test, thereby failing assertions. Current diff disables ASH for certain tests. Jira: DB-13182 Test Plan: ./yb_build.sh --java-test 'org.yb.pgsql.TestYbQueryDiagnostics#testPgssResetBetweenDiagnostics' Reviewers: asaha Reviewed By: asaha Differential Revision: https://phorge.dev.yugabyte.com/D40126 * [#25004] YSQL: Fix the backend type of the physical connection in ysql conn mgr Summary: Post pg15 merge into the master, the type/name of backends created for physical connections in transaction pool are coming to be client backend instead of yb-conn-mgr worker connection. This diff fixes this issue. Fix Based on value of the guc variable `yb_is_client_ysqlconnmgr`, the type of backend is set to yb-conn-mgr worker connection or client backend. While parsing the startup packet of the transactional backend process, the guc variables coming in the startup packet are not yet processed (they are processed in later stage of initialization of the backend). Therefore, in this diff during parsing of the startup packet itself, `yb_is_client_ysqlconnmgr` guc variable is been set and based on it's correct value the type of backend is assigned. This diff also fixes one of implementation gap while making the connections sticky when client executes SET stmt to set "session_authorization" and "role" guc variables. Jira: DB-14148 Test Plan: Jenkins: enable connection manager, all tests Reviewers: skumar, rbarigidad, vpatibandla, stiwary Reviewed By: stiwary Subscribers: svc_phabricator, yql Differential Revision: https://phorge.dev.yugabyte.com/D40323 * [#25094] yugabyted: Fix yugabyted UI slow queries page due to pg15 pg_stat_statements changes Summary: The query used to get slow queries in the yugabyted UI API server queries `pg_stat_statements`. Some of the column names were changed from `total_time, min_time, max_time, mean_time, stddev_time` to `total_exec_time, min_exec_time, max_exec_time, mean_exec_time, stddev_exec_time` in pg15 and need to be changed in the query as well. Jira: DB-14240 Test Plan: manual tests Reviewers: sgarg-yb, nikhil Reviewed By: sgarg-yb Subscribers: yugabyted-dev, djiang Differential Revision: https://phorge.dev.yugabyte.com/D40303 * [#25118] DocDB: Fix org.yb.pgsql.TestPgRegressThirdPartyExtensionsPgvector#schedule Summary: Commit bbf8c1288a87a15be2913fb1a665c0740b9de85c/D40131 contains debug leftover with CHECK, that is causing ybdummyann tests to fail. Fixed by removed this check and adjusting logic in PgsqlReadOperation::ExecuteVectorSearch. Jira: DB-14263 Test Plan: org.yb.pgsql.TestPgRegressThirdPartyExtensionsPgvector#schedule Reviewers: slingam, arybochkin Reviewed By: slingam, arybochkin Subscribers: svc_phabricator, ybase, yql Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D40336 * [#24647] YSQL: In major catalog upgrade, avoid reading current version catalog before it's valid Summary: During normal operation of a YB universe, the YB master itself sometimes references PG catalog tables. For example, both during CreateTable and as a periodic refresh, it references pg_tablespace, and as a periodic task it reads from pg_database. In the existing code, during a ysql major catalog upgrade, the master will generate table UUIDs for the current ysql major catalog version for these tables, even though they may not exist or be fully populated yet. Modify the YB master code to read from prior ysql major catalog version tables until the current ysql major catalog version tables are valid. Note that as a side effect, this allows us to consolidate and simplify the logic to read from the correct table to retrieve the current per-db catalog versions (not to be confused with "ysql major catalog version," our terminology for PG11 or PG15) during the upgrade. After this commit, you may need to reinitdb, if you haven't done so in a while, to pick up a catalog snapshot that correctly holds the current version in ysql_major_catalog_upgrade_info.catalog_version. Jira: DB-13711 Test Plan: Jenkins On MacOS 14 arm64: ./yb_build.sh release --cxx-test integration-tests_basic_upgrade-test --gtest_filter UpgradeFrom_2_25_0_0/BasicUpgradeTest.TestUpgrade/0 --with_tests ./yb_build.sh fastdebug --cxx-test integration-tests_basic_upgrade-test --gtest_filter UpgradeFrom_2_25_0_0/BasicUpgradeTest.TestUpgrade/0 --with_tests (to test builds.xml builds for Mac debug, primarily, because that build flavor isn't exercised by the automated tests right now) Reviewers: hsunder Reviewed By: hsunder Subscribers: hsunder, smishra, ybase, yql Differential Revision: https://phorge.dev.yugabyte.com/D39813 * [#25129] DocDB: Increase PgPartmanTest test timeout in ASAN mode Summary: PgPartmanTest performs hundreds of table modification tasks, it takes significant amount of time especially in ASAN mode. So those tests does not fit into default timeout: PgPartmanTest.TestIdTimeSubpartNative PgPartmanTest.TestIdTimeSubpartCustomStartNative Increased timeout to 1200 seconds in ASAN mode Jira: DB-14279 Test Plan: ./yb_build.sh asan --gtest_filter PgPartmanTest.TestIdTimeSubpartNative ./yb_build.sh asan --gtest_filter PgPartmanTest.TestIdTimeSubpartCustomStartNative Reviewers: devansh.saxena, skumar Reviewed By: devansh.saxena Subscribers: yql, ybase Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D40356 * [#23901] YSQL, ASH: Sample background workers Summary: This diff adds support for sampling YSQL background workers. Summary of changes - - Add a new const query id for YSQL background workers - While deciding whether to consider a backend for sampling or not, we now only check for `MyProc->yb_is_ash_metadata_set` - The CPU wait event name in YSQL is changed from `QueryProcessing` to `OnCpu_Active` because now we also have background workers which may not be executing queries - We no longer take locks while setting root_request_id, client_node_ip, pid, more details in code comments - We don't sample the ASH collector itself because it will always show CPU events - kQueryIdForCatalogRequests is renamed to kQueryIdForUncomputedQueryId as this is more accurate name - Two sync RPCs CronGetLastMinute and CronSetLastMinute called from the pg_cron background worker are now also instrumented **Upgrade/Rollback safety:** Safe to upgrade/rollback, if ASH metadata is not found in the PB, then metadata in the threadlocal wait state object is not updated. If ASH metadata is sent, and the other tserver is still on old version, it will simply ignore and continue doing it's work Jira: DB-12805 Test Plan: ./yb_build.sh --cxx-test pg_ash-test --gtest_filter PgBgWorkersTest.ValidateBgWorkers ./yb_build.sh --cxx-test pg_ash-test --gtest_filter PgWaitEventAuxTest.PgCronRPCs Reviewers: hsunder Reviewed By: hsunder Subscribers: smishra, yql, ybase Differential Revision: https://phorge.dev.yugabyte.com/D39062 * [DOCS] python Smart Driver (psycopg2) docs update for RR support (#25074) * [#25119] Revert `yb_amiscoveredbymaintable` related changes Summary: This reverts changes related to `yb_amiscoveredbymaintable` in b4ef0c523359e3572c2bc3f619780cacf6edbbde and .b9b57c6f791cb9f28d8bec9372e9a433a903980c in line with the most up to date requirements for index co-partitioning. In the previous design, co-partitioned indexes were meant to share almost all metadata and storage with the main table. The currently implemented design for co-partitioning in docdb creates such indexes a different way. Co-partitioned indexes on creation create new data storage but get colocated with their respective main tables in a mechanism similar to the existing colocation index creation. This change reverts any remnants of the previous design on the query layer to allow for this redesign. b9b57c6f791cb9f28d8bec9372e9a433a903980c had a few conflicts with this change that modified some call-sites of `YBIsCoveredByMainTable`. Test Plan: Jenkins Reviewers: telgersma, jason, fizaa Reviewed By: telgersma Subscribers: yql, ybase Differential Revision: https://phorge.dev.yugabyte.com/D40340 * Revert "[PLAT-13112]Block deletion of in use-releases" Summary: This reverts commit af1bdd97294f8bda926ae38c1a08186b0d419d24/D37451 since it broke UTs. Will be fixed by @anabaria once he is back from PTO. Test Plan: N/A Reviewers: anabaria Reviewed By: anabaria Subscribers: anabaria, yugaware Differential Revision: https://phorge.dev.yugabyte.com/D40373 * [DOC-566] TA-23476 Update Tech advisory (#25091) * update existing ta-23476 * add shortcode for versions * Apply suggestions from review Co-authored-by: Aishwarya Chakravarthy <[email protected]> * use bullet points * update TA 23476 on TA landing page. * add timestamp example --------- Co-authored-by: Aishwarya Chakravarthy <[email protected]> * [#22518] xCluster: Fix XClusterSafeTimeTest.ConsumerHistoryCutoff Summary: Fixed a couple of bug in VerifyHistoryCutoffTime to make test pass. Jira: DB-11444 Test Plan: ./yb_build.sh fastdebug --gcc11 --cxx-test xcluster_safe_time-itest --gtest_filter XClusterSafeTimeTest.ConsumerHistoryCutoff -n 400 -- -p 8 Reviewers: hsunder, xCluster Reviewed By: hsunder Subscribers: ybase Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D40375 * [#25130] DocDB: Preventing altering table that is being prepared Summary: Alter table request could arrive while table is being prepared. It could ruin table states transition logic. Fixed by rejecting alter table while table is being prepared. Also removed explicit vmodule setup in tests. If it is necessary to execute test with vmodule settings we could always use `--test-args --vmodule=...` way. Which is blocked by explicit value for vmodule flag. Jira: DB-14280 Test Plan: ./yb_build.sh fastdebug --gcc11 --gtest_filter CDCSDKYsqlTest.TestAddManyColocatedTablesOnNamesapceWithStream -n 40 Reviewers: zdrudi, xCluster, hsunder Reviewed By: zdrudi Subscribers: ybase, yql Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D40367 * [#17898] DocDB: Fixed bug with retaining DelayedTask during shutdown Summary: DelayedTask::Run checks whether Reactor started shutdown and just exit. Because of this this task does not invoke callback and hangs forever in messenger. Actually we don't need this check, since there are a more robust checks in our code to detect whether task should be aborted. So removed this check. Jira: DB-6980 Test Plan: ./yb_build.sh fastdebug --gcc11 --gtest_filter MasterTest.TestRegisterDistBroadcastDupPrivateUsePrivateIpNeverCheckUsed -n 400 Reviewers: hsunder Reviewed By: hsunder Subscribers: ybase Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D40364 * [DOC-554][CLOUDGA-24513][ybm] VictoriaMetrics GA (#25056) * YBM VictoriaMetrics GA * edit * review comments * [doc][ybm] Change log for Victoriametrics ga (#25134) * change log for victoriametrics ga * format * [#24324][conn-mgr] Add logical client version matching to select backends Summary: This diff adds the logic of selecting transactional backends based on matching the logical client version of the server to that of the logical client. **This feature is guarded by preview flag** `ysql_conn_mgr_version_matching`. There is one more preview flag is added named `ysql_conn_mgr_version_matching_connect_higher_version` . Use case of this flag is discussed later in algorithm. Algorithm description in chronological order. 1.) When a backend spawns, it reads the current_version from pg_yb_logical_client_version table added as part of D38335 and sends it to frontend as parameter status packet. 2.) At frontend, `od_backend_startup` in backend.c, this parameter status packet is parsed and stored in server->logical_client_version. 3.) When a new logical client comes up, it is authenticated by auth backend. After successful authentication, it stores it's logical_client_version from auth backend's logical client version. 4.) After authentication , during routing of the client for getting transactional backend, we first check the `logical_client_version` of the client to that of route's `max_logical_client_version`. - If former is greater than later, then we - update the route's `max_logical_client_version` - mark all sever's in active and idle pool's `marked_for_close` as true. - If the logical_client_version of client is less than route's max_logical_client_version then we return `OD_ROUTER_ERROR` leading to disconnect of the server. - After that we select a backend from idle pool having these two things: 5.) Now there would be two cases for the server selection algorithm. Either we get a server or we get NULL. - If we get a server , we simply attach - If not, we spawn a new server and check the logical_client_version of that sever to the client's logical client version. There could be a case where attached server's version is greater than client's logical client version. In this case there if `ysql_conn_mgr_version_matching_connect_higher_version` is: - Set to false: return `OD_ESERVER_CONNECT` - Set to true: return `OD_OK`; Test Plan: Jenkins: enable connection manager, all tests ./yb_build.sh release --java-test org.yb.ysqlconnmgr.TestLogicalClientVersion Reviewers: skumar, stiwary, mkumar, rbarigidad Reviewed By: rbarigidad Subscribers: svc_phabricator, yql Differential Revision: https://phorge.dev.yugabyte.com/D38749 * [#25030] YCQL: Replace autoflag ycql_suppress_group_by_error with a regular gFlag Summary: In commit 80def06f8c19ad7cbc52f41b4be48d158157a418, an autoflag `ycql_suppress_group_by_error` was introduced to suppress the error raised when using the `GROUP BY` clause. 1. This should not have been an autoflag because we want users to decide this behavior. - A new regular gFlag, `ycql_ignore_group_by_error`, has been introduced for future use since autoflags cannot be demoted or removed. - When set to true(default), no error is raised, and the `GROUP BY` clause is ignored. When set to false, the error is raised as usual. 2. The autoflag definition was previously in pt_select.cc. Its definition has now been moved to server_common_flags.cc because, due to differences in the linking scheme for various build types, this flag was not available in master flags for certain build types. - The flag does not determine/change any behaviour. Jira: DB-14168 Test Plan: ./yb_build.sh --java-test org.yb.cql.TestCreateTable#testCreateTableWithColumnNamedGroup ./yb_build.sh --java-test org.yb.cql.TestSelect#testGroupBy Reviewers: stiwary, hsunder, skumar Reviewed By: stiwary, hsunder Subscribers: yql, ybase Differential Revision: https://phorge.dev.yugabyte.com/D40327 * [#25136] YSQL: Decrease number of iteration in PgLibPqTest.MasterRestartReadPastGlobalLimit TSAN mode Summary: The test performs 10k iterations, in TSAN mode it takes 0.1s to perform single iteration in TSAN mode. So test just times out because 15 minutes is required to complete such number of iterations. Decreased number of iterations to 1k for sanitizers. Jira: DB-14286 Test Plan: ./yb_build.sh tsan --gtest_filter PgLibPqTest.MasterRestartReadPastGlobalLimit -n 40 Reviewers: jason, patnaik.balivada Reviewed By: patnaik.balivada Subscribers: ybase, yql Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D40381 * [#23331] yugabyted: Fixing yugabyted tests broken with pg_isready diff D34597. Summary: On Jenkins pipelines `pg_isready` binary is not available in the paths configured in yugabyted. Adding required changes to find the `pg_isready` on Jenkins VMs. Jira: DB-12256 Test Plan: ./yb_build.sh --java-test 'org.yb.yugabyted.TestYugabytedSingleNode' Reviewers: sgarg-yb Reviewed By: sgarg-yb Subscribers: yugabyted-dev Differential Revision: https://phorge.dev.yugabyte.com/D40379 * [DB-13952]yugabyted: Fixing the count of automatic, manual refactoring. Summary: Schema analysis API data usage - - data for `AUTOMATIC DDL IMPORT` `MANUAL REFACTORING` `TOTAL ANALYZED` is being read from `current_analysis_report.recommended_refactoring.refactor_details.manual/automatic` - For the summary table, values are being read from `current_analysis_report.recommended_refactoring.refactor_details` list. - For both UI sections, using `refactor_details.automatic` value requires an API change. Automatic value is being calculated by subtracting the count of conversion issues instead of subtracting the count of `sql_objects.invalidCount`. Jira: DB-13952 Test Plan: Manual Test Reviewers: sgarg-yb Reviewed By: sgarg-yb Differential Revision: https://phorge.dev.yugabyte.com/D40328 * [#24697] Docdb: Table locks - support releasing locks for a session/host Summary: Api to allow releasing locks for a session/host when either a) The session ends, or b) The tserver loses its lease/restarts. The TServer should detect when a session ends, and may call the API to release all locks held by a particular session. Once YSQL lease is implemented, the master may realize that a TServer has lost it's lease, or may discover that the TServer has had a restart when a TServer shows up with a new instance_seqno. A new incarnation-id may be generated, and then a background task clears out the locks held by previous incarnations. Other changes: - Try to make Unlock/Lock apis take a callback instead Jira: DB-13773 Test Plan: yb_build.sh --cxx-test object_lock-test Reviewers: zdrudi, bkolagani Reviewed By: zdrudi Subscribers: ybase Differential Revision: https://phorge.dev.yugabyte.com/D39749 * [PLAT-15959]: Update PDB policy in case of pods change in K8s universe Summary: This diff is in continuation of PDB support: D39864 Added changes to update PDB policy in case of edit universe or edit read-replica cluster. Test Plan: Tested manually by performing actions like pod change and create/delete read-replica. Reviewers: sneelakantan, anijhawan, vkumar Reviewed By: vkumar Subscribers: yugaware Differential Revision: https://phorge.dev.yugabyte.com/D40315 * [DB-13952]yugabyted: Fixing the ui changes after the addition of invalid obj count. Summary: Addition of invalid object count column on assessment and analysis voyager page. Jira: DB-13952 Test Plan: Manual testing Reviewers: nikhil Reviewed By: nikhil Subscribers: yugabyted-dev, krishna.tripathi Differential Revision: https://phorge.dev.yugabyte.com/D40387 * [docs] Release notes for 2.20.8.0-b53 (#25079) * rn for 2.20.8.0 * changed date * [#25089] build: Allow building with artifacts from thirdparty PR Summary: This diff makes the following changes to improve workflow of making thirdparty changes: - Allow specifying GitHub artifact IDs for thirdparty builds and their checksums in thirdparty_archives.yml instead of tags, e.g. ``` - os_type: ubuntu22.04 architecture: x86_64 compiler_type: gcc11 release_artifact_id: 1234567890 checksum_artifact_id: 1234567891 ``` Artifact downloads are structured as a zip file containing archive.tar.gz/archive.tar.gz.sha256. We cache only the extracted archive.tar.gz/archive.tar.gz.sha256, renamed as github-artifact-RELEASE_ARTIFACT_ID.tar.gz/github-artifact-RELEASE_ARTIFACT_ID.tar.gz.sha256. The build is extracted to /opt/yb-build/thirdparty/RELEASE_ARTIFACT_ID, and a symlink is created at /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v.... pointing to /opt/yb-build/thirdparty/RELEASE_ARTIFACT_ID (this is needed because thirdparty binaries used in the build have this path hardcoded in rpath). - Make thirdparty_tool be able to pull thirdparty builds from a thirdparty PR instead of scanning release tags, e.g. ``` ./build-support/thirdparty_tool -u --thirdparty-pr 12345678 ``` will generate thirdparty_archives.yml using artifacts from the latest version of PR #12345678 on the thirdparty repo. Accessing artifacts requires a GitHub token to be set (either via the GITHUB_TOKEN environmental variable, or in a file specified in the YB_GITHUB_TOKEN_FILE_PATH environmental variable); one of the two must be set when building with artifacts from a thirdparty PR. Some old linuxbrew handling code was also removed. Jira: DB-14217 Test Plan: Jenkins: compile only Tested thirdparty_tool with and without the new `--thirdparty-pr` option, and tested local builds in both setups. Reviewers: steve.varnau Reviewed By: steve.varnau Subscribers: ybase Differential Revision: https://phorge.dev.yugabyte.com/D40118 * [#25053] YSQL: Fix TestPgDdlConcurrency for TSAN build Summary: Similar fix already made for ASAN build. The test still used 50 thread for TSAN build. It was reported that 50 threads in TSAN build caused other CPU intensive tests on the same test VM to fail. Reduced to 10 threads for TSAN build as well. The test runs more flaky under TSAN build with this change, I will look into how to get rid of the flakiness separately. Jira: DB-14184 Test Plan: ./yb_build.sh tsan --java-test 'org.yb.pgsql.TestPgDdlConcurrency#testModifiedTableWrite' -n 20 --tp 1 Reviewers: kfranz, mihnea Reviewed By: kfranz Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D40398 * [#24707] YSQL: pg_cron: Kill inflight jobs when leader changes Summary: Kill jobs if we are no longer leader. If there is a long running job when the cron leader moved due to load balance or upgrade activities, this will ensure that there is only one occurrence of the job running at any given time. Fixes #24707 Jira: DB-13784 Test Plan: PgCronTest.CancelJobOnLeaderChange Reviewers: tnayak, fizaa Reviewed By: tnayak Subscribers: ybase, yql Differential Revision: https://phorge.dev.yugabyte.com/D40335 * [#25145] DocDB: Fix MasterTabletServer::GetServingTablet Summary: Missing null check. `CatalogManager` is yb-masters `tserver::TabletPeerLookupIf`, so moving the inheritance to `CatalogManagerIf` and using it in `MasterTabletServer`. The `CatalogManager` performs all the required checks correctly. Fixes #25145 Jira: DB-14305 Test Plan: Jenkins Reviewers: bkolagani Reviewed By: bkolagani Subscribers: ybase Differential Revision: https://phorge.dev.yugabyte.com/D40405 * [PLAT-16129][YBA CLI] RBAC APIs - 1 Summary: Add commands corresponding to RBAC APIs. RBAC Command ``` yba rbac Manage YugabyteDB Anywhere RBAC (Role-Based Access Control) Usage: yba rbac [flags] yba rbac [command] Available Commands: permission Manage YugabyteDB Anywhere RBAC permissions role Manage YugabyteDB Anywhere RBAC roles Flags: -h, --help help for rbac Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` Permission command ``` yba rbac permission Manage YugabyteDB Anywhere RBAC permissions Usage: yba rbac permission [flags] yba rbac permission [command] Available Commands: describe Describe a YugabyteDB Anywhere RBAC permission list List YugabyteDB Anywhere permissions Flags: -h, --help help for permission Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) Use "yba rbac permission [command] --help" for more information about a command. ``` List Permission ``` yba rbac permission list List YugabyteDB Anywhere permissions Usage: yba rbac permission list [flags] Aliases: list, ls Examples: yba rbac permission list Flags: -n, --name string [Optional] Name of the permission. Quote name if it contains space. --resource-type string [Optional] Resource type of the permission. Allowed values: universe, role, user, other. If not specified, all resource types are returned. -h, --help help for list Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` Describe Persmiison ``` yba rbac permission describe -h Describe a RBAC permission in YugabyteDB Anywhere Usage: yba rbac permission describe [flags] Aliases: describe, get Examples: yba rbac permission describe --name <permission-name> Flags: -n, --name string [Required] Name of the permission. Quote name if it contains space. -h, --help help for describe Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` Role command ``` yba rbac role Manage YugabyteDB Anywhere RBAC roles Usage: yba rbac role [flags] yba rbac role [command] Available Commands: create Create YugabyteDB Anywhere RBAC roles delete Delete a YugabyteDB Anywhere role describe Describe a YugabyteDB Anywhere RBAC role list List YugabyteDB Anywhere roles update Update a YugabyteDB Anywhere role Flags: -h, --help help for role Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` List Role command ``` yba rbac role list List YugabyteDB Anywhere roles Usage: yba rbac role list [flags] Aliases: list, ls Examples: yba rbac role list Flags: -n, --name string [Optional] Name of the role. Quote name if it contains space. --type string [Optional] Role type. Allowed values: system, custom. If not specified, all role types are returned. -h, --help help for list Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` Describe role: ``` yba rbac role describe -h Describe a RBAC role in YugabyteDB Anywhere Usage: yba rbac role describe [flags] Aliases: describe, get Examples: yba rbac role describe --name <role-name> Flags: -n, --name string [Required] Name of the role. Quote name if it contains space. -h, --help help for describe Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` Delete role ``` yba rbac role delete -h Delete a role in YugabyteDB Anywhere Usage: yba rbac role delete [flags] Aliases: delete, remove, rm Examples: yba role delete --name <role-name> Flags: -n, --name string [Required] The name of the role to be deleted. -f, --force [Optional] Bypass the prompt for non-interactive usage. -h, --help help for delete Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) Create role ``` yba rbac role create -h Create YugabyteDB Anywhere RBAC roles Usage: yba rbac role create [flags] Aliases: create, add Examples: yba rbac role create --name <role-name> \ --permission resource-type=other::action=read \ --description <description> Flags: -n, --name string [Required] Name of the role. Quote name if it contains space. --description string [Optional] Description of the role. Quote description if it contains space. --permission stringArray [Required] Permissions associated with the role. Minimum number of required permissions = 1. Provide the following double colon (::) separated fields as key-value pairs: "resource-type=<resource-type>::action=<action>". Both are requires key-values. Allowed resource types are universe, role, user, other. Allowed actions are create, read, update, delete, pause_resume, backup_restore, update_role_bindings, update_profile, super_admin_actions, xcluster. Each permission needs to be added using a separate --permission flag. Example: --permission resource-type=other::action=delete --permission resource-type=universe::action=write -h, --help help for create Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` Update role command: ``` yba rbac role update -h Update a role in YugabyteDB Anywhere Usage: yba rbac role update [flags] Aliases: update, edit Examples: yba rbac role update --name <role-name> \ --add-permission resource-type=other::action=create Flags: -n, --name string [Required] Role name to be updated. --add-permission stringArray [Optional] Add permissions to the role. Provide the following double colon (::) separated fields as key-value pairs: "resource-type=<resource-type>::action=<action>". Both are requires key-values. Allowed resource types are: universe, role, user, other. Allowed actions are: create, read, update, delete, pause_resume, backup_restore, update_role_bindings, update_profile, super_admin_actions, xcluster.Quote action if it contains space. Each permission needs to be added using a separate --add-permission flag. --remove-permission stringArray [Optional] Remove permissions from the role. Provide the following double colon (::) separated fields as key-value pairs: "resource-type=<resource-type>::action=<action>". Both are requires key-values. Allowed resource types are: universe, role, user, other. Allowed actions are: create, read, update, delete, pause_resume, backup_restore, update_role_bindings, update_profile, super_admin_actions, xcluster.Quote action if it contains space. Each permission needs to be removed using a separate --remove-permission flag. -h, --help help for update Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` Test Plan: `yba rbac permission list` ``` Name Resource Type Action Permission Valid On Resource Update Role ROLE UPDATE false Create Role ROLE CREATE false View Role ROLE READ false Delete Role ROLE DELETE false Delete User USER DELETE false Update Role Bindings USER UPDATE_ROLE_BINDINGS false Create User USER CREATE false Update User Profile USER UPDATE_PROFILE false View User USER READ false View Universe UNIVERSE READ true Manage XCluster UNIVERSE XCLUSTER true Update Universe UNIVERSE UPDATE true Create Universe UNIVERSE CREATE false Backup/Restore Universe UNIVERSE BACKUP_RESTORE true Pause/Resume Universe UNIVERSE PAUSE_RESUME true Delete Universe UNIVERSE DELETE true View Resource OTHER READ false Create Resource OTHER CREATE false Update Resource OTHER UPDATE false Delete Resource OTHER DELETE false Super Admin Actions OTHER SUPER_ADMIN_ACTIONS false ``` `yba rbac permission describe --name "View Universe"` ``` General Name Resource Type Action Permission Valid On Resource View Universe UNIVERSE READ true Permission Details Description Allows user to view a universe. Prerequisite Permissions Permission 1: Details Action Resource Type READ OTHER ``` `yba rbac role list` ``` Name UUID Role Type ReadOnly a392c6cc-a57d-4a04-acb8-0e1d4d1f1205 System BackupAdmin f201e3fc-845a-4b5f-9f84-052b6c24d1a3 System ConnectOnly 7b10ff16-5ac0-4cfe-b1ee-0a0673a5f86b System Admin 1c4f107e-a2a4-4b45-916b-3ea6a936e85e System SuperAdmin 5853e7a7-0a89-4472-a354-d473de7c21ae System Developer L2 bef2eb73-020a-45f1-a9c2-23a5877714d5 Custom Software L1 019e26b0-0ee2-476c-bb86-b85c20c26412 Custom ``` `yba rbac role get -n "Developer L2"` ``` General Name UUID Role Type Developer L2 bef2eb73-020a-45f1-a9c2-23a5877714d5 Custom Role Details Description Access to selected universes Created On Updated On Thu, 08 Feb 2024 03:12:41 +0000 Mon, 05 Aug 2024 05:49:24 +0000 Permissions Permission 1: Details Action Resource Type BACKUP_RESTORE UNIVERSE Permission 2: Details Action Resource Type READ UNIVERSE Permission 3: Details Action Resource Type PAUSE_RESUME UNIVERSE Permission 4: Details Action Resource Type READ OTHER Permission 5: Details Action Resource Type DELETE UNIVERSE Permission 6: Details Action Resource Type XCLUSTER UNIVERSE Permission 7: Details Action Resource Type UPDATE UNIVERSE ``` `yba rbac role create -n test-cli --permission resource-type=universe::action=delete --permission resource-type=universe::action=read --permission resource-type=other::action=read` ``` Name UUID Role Type test-cli f323efd8-90ea-4090-9b41-00922866aea0 Custom ``` `yba rbac role update -n test-cli --add-permission resource-type=universe::action=read` ``` Name UUID Role Type test-cli f323efd8-90ea-4090-9b41-00922866aea0 Custom ``` Reviewers: skurapati Reviewed By: skurapati Subscribers: yugaware Differential Revision: https://phorge.dev.yugabyte.com/D40222 * [#23331] yugabyted: Update directory location of pg_isready binary in Development Environemnt. Summary: Updating the directory location of pg_isready binary from `build/clang17-dynamic-ninja` to `build/latest`. Jira: DB-12256 Test Plan: Manual Tests Reviewers: sgarg-yb, djiang Reviewed By: djiang Subscribers: yugabyted-dev Differential Revision: https://phorge.dev.yugabyte.com/D40396 * [PLAT-16128]User management commands - update profile, reset password Summary: Password reset for logged in user: ``` yba user reset-password -h Reset password of currently logged in user in YugabyteDB Anywhere Usage: yba user reset-password [flags] Aliases: reset-password, reset Examples: yba user reset-password \ --current-password <current-password> --new-password <new-password> Flags: --current-password string [Required] The current password of the user. --new-password string [Required] The new password of the user. -h, --help help for reset-password Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` Update user ``` yba user update Update a user in YugabyteDB Anywhere Usage: yba user update [flags] Aliases: update, edit Examples: yba user update --email <user-email> --timezone "America/Los_Angeles" Flags: -e, --email string [Required] The email of the user to be updated. --timezone string [Optional] The timezone of the user to be updated. -h, --help help for update Global Flags: -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` Test Plan: `yba user reset-password --current-password Password#123 --new-password Password#1234` ``` Email UUID …
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-14286
Description
Test times out
Issue Type
kind/failing-test
Warning: Please confirm that this issue does not contain any sensitive information
The text was updated successfully, but these errors were encountered: