Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YSQL][ASH] Create a function and view yb_wait_event_desc #19131

Closed
1 task done
hbhanawat opened this issue Sep 17, 2023 · 0 comments
Closed
1 task done

[YSQL][ASH] Create a function and view yb_wait_event_desc #19131

hbhanawat opened this issue Sep 17, 2023 · 0 comments
Assignees
@abhinab-yb
Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@hbhanawat
Copy link
Contributor

hbhanawat commented Sep 17, 2023 •
edited by jira bot
Loading

Jira Link: DB-7929

Description

- List and detailed description of wait events. This can be joined with yb_active_universe_history on wait_event_code to get a description of wait events. This is a usability thing so that users understand what a wait event means.
- Ensure that the view is part of the system view

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@hbhanawat hbhanawat added area/ysql Yugabyte SQL (YSQL) status/awaiting-triage Issue awaiting triage area/active-history Active history of sessions labels Sep 17, 2023
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels Sep 17, 2023
@hbhanawat hbhanawat mentioned this issue Sep 17, 2023
Open
55 tasks
@yugabyte-ci yugabyte-ci removed area/active-history Active history of sessions status/awaiting-triage Issue awaiting triage labels Sep 17, 2023
@hbhanawat hbhanawat added the area/active-history Active history of sessions label Sep 18, 2023
@hbhanawat hbhanawat changed the title [YSQL][ActiveHistory] Create a function and view yb_wait_event_desc [YSQL][ASH] Create a function and view yb_wait_event_desc Jan 3, 2024
@yugabyte-ci yugabyte-ci removed the area/active-history Active history of sessions label Jun 13, 2024
abhinab-yb added a commit that referenced this issue Aug 20, 2024
@abhinab-yb
[#19131] YSQL, ASH: Create a view to display wait events information
1a3c113 
Summary:
This diff introduces a view `yb_wait_event_desc` to display the
information related to each wait event. The component is not
included in the view because some wait events can be found
in multiple components.

This diff only covers the wait events introduced by YB, the PG
wait events will be covered in a future diff.

This diff also does some minor refactoring
- Deprecate kRpcs_WaitOnMutexInShutdown which was not used anywhere in the code
- Rename YCQLQueryProcessing to YCQLQuery, as it's YSQL counterpart is YSQLQuery

```
yugabyte=# select * from yb_wait_event_desc;
 wait_event_class | wait_event_type |                wait_event                |                                            wait_event_description
------------------+-----------------+------------------------------------------+--------------------------------------------------------------------------------------------------------------
 TServerWait      | Network         | CatalogRead                              | A YSQL backend is waiting for a catalog read from master.
 TServerWait      | Network         | IndexRead                                | A YSQL backend is waiting for a secondary index read from DocDB.
 TServerWait      | Network         | StorageRead                              | A YSQL backend is waiting for a table read from DocDB.
 TServerWait      | Network         | StorageFlush                             | A YSQL backend is waiting for a table/index read/write from DocDB.
 Common           | Cpu             | OnCpu_Active                             | A rpc/task is being actively processed on a thread.
 Common           | Cpu             | OnCpu_Passive                            | A rpc/task is waiting for a thread to pick it up.
 Common           | WaitOnCondition | Idle                                     | The raft log appender/sync thread is idle.
 Common           | WaitOnCondition | Rpc_Done                                 | A rpc is done and waiting for the reactor to send the response to a YSQL/YCQL backend.
 Common           | DiskIO          | RetryableRequests_SaveToDisk             | The in-memory state of the retryable requests is being saved to the disk.                                   +
                  |                 |                                          | This generally happens in the background during a WAL log roll, or Remote bootstrap.
 TabletWait       | WaitOnCondition | MVCC_WaitForSafeTime                     | A read/write rpc is waiting for the SafeTime to be at least the desired read-time.
 TabletWait       | WaitOnCondition | LockedBatchEntry_Lock                    | A read/write rpc is waiting for a DocDB row level lock.
 TabletWait       | WaitOnCondition | BackfillIndex_WaitForAFreeSlot           | A backfill index rpc is waiting for a slot to open if there are too many backfill requests at the same time.
 TabletWait       | DiskIO          | CreatingNewTablet                        | The CreateTablet RPC is creating a new tablet, this may involve writing metadata files, causing I/O wait.
 TabletWait       | DiskIO          | SaveRaftGroupMetadataToDisk              | The raft/tablet metadata is being written to disk. Generally during Snapshot/Restore operations.
 TabletWait       | Network         | TransactionStatusCache_DoGetCommitData   | A rpc needs to look up the commit status of a particular transaction.
 TabletWait       | WaitOnCondition | WaitForYSQLBackendsCatalogVersion        | CREATE INDEX is waiting for YSQL backends to have up-to-date pg_catalog.
 TabletWait       | DiskIO          | WriteSysCatalogSnapshotToDisk            | Writing initial sys catalog snapshot during initdb.
 TabletWait       | WaitOnCondition | DumpRunningRpc_WaitOnReactor             | DumpRunningRpcs is waiting on reactor threads.
 TabletWait       | Network         | ConflictResolution_ResolveConficts       | A read/write rpc is waiting to identify conflicting transactions.
 TabletWait       | WaitOnCondition | ConflictResolution_WaitOnConflictingTxns | A read/write rpc is waiting for conflicting transactions to complete.
 Consensus        | Network         | Raft_WaitingForReplication               | A write rpc is waiting for raft replication.
 Consensus        | Cpu             | Raft_ApplyingEdits                       | A write rpc is applying Raft edits locally.
 Consensus        | DiskIO          | WAL_Append                               | A write rpc is Persisting WAL edits.
 Consensus        | DiskIO          | WAL_Sync                                 | A write rpc is Synchronizing WAL edits.
 Consensus        | DiskIO          | ConsensusMeta_Flush                      | ConsensusMetadata is flushed, say during Raft term/config change or Remote bootstrap etc.
 Consensus        | WaitOnCondition | ReplicaState_TakeUpdateLock              | A write/alter RPC needs to wait for the ReplicaState lock to replicate a batch of writes through Raft.
 RocksDB          | DiskIO          | RocksDB_ReadBlockFromFile                | RocksDB is reading a block from a file.
 RocksDB          | DiskIO          | RocksDB_OpenFile                         | RocksDB is opening a file.
 RocksDB          | DiskIO          | RocksDB_WriteToFile                      | RocksDB is writing to a file.
 RocksDB          | Cpu             | RocksDB_Flush                            | RocksDB is doing a flush.
 RocksDB          | Cpu             | RocksDB_Compaction                       | RocksDB is doing a compaction.
 RocksDB          | WaitOnCondition | RocksDB_PriorityThreadPoolTaskPaused     | RocksDB paused one of flush/compaction tasks for another one with a higher priority
 RocksDB          | DiskIO          | RocksDB_CloseFile                        | RocksDB is closing a file.
 RocksDB          | WaitOnCondition | RocksDB_RateLimiter                      | RocksDB flush/compaction is slowing down due to rate limiter throttling access to disk.
 RocksDB          | WaitOnCondition | RocksDB_WaitForSubcompaction             | RocksDB is waiting for a compaction to complete.
 RocksDB          | DiskIO          | RocksDB_NewIterator                      | RocksDB is waiting for a new iterator to be created
 YCQLQuery        | Cpu             | YCQL_Parse                               | YCQL is parsing a query.
 YCQLQuery        | Cpu             | YCQL_Read                                | YCQL is processing a read query.
 YCQLQuery        | Cpu             | YCQL_Write                               | YCQL is processing a write query.
 YCQLQuery        | Cpu             | YCQL_Analyze                             | YCQL is analyzing a query.
 YCQLQuery        | Cpu             | YCQL_Execute                             | YCQL is executing a query.
 Client           | Network         | YBClient_WaitingOnDocDB                  | YB Client is waiting on DocDB to return a response.
 Client           | Network         | YBClient_LookingUpTablet                 | YB Client is looking up tablet information from the master.
(43 rows)
```
Jira: DB-7929

Test Plan: ./yb_build.sh --java-test TestYsqlUpgrade

Reviewers: jason, hbhanawat, amitanand

Reviewed By: jason

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D37301
jasonyb pushed a commit that referenced this issue Aug 21, 2024
@yugabyte-ci @jaki
[BACKPORT pg15-cherrypicks] all: Bulk port from master - 81
194bc85 
Summary:
 feb7ddd [#23524] DocDB: Implement VectorIndexFetcher by VectorIndexUpdate
 5b21f96 Add content for gin_trgm_ops (#23528)
 5878518 [docs] renamed references (#23212)
 67147d2 [PLAT-14993] Clean out pg_data_11 dir on onprem node release/delete
 Excluded: 6daf129 [#23557] Remove empty CatCacheIdMisses_End metric
 d0d8535 [PLAT-14960] make ssh key optional for onprem manual provisioning
 54776ab [DB-10610] YSQL: import Detect integer overflow while computing new array dimensions
 Excluded: 1a3c113 [#19131] YSQL, ASH: Create a view to display wait events information

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: jason, tfoucher

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D37421
abhinab-yb added a commit that referenced this issue Aug 22, 2024
@abhinab-yb
[BACKPORT pg15-cherrypicks][#19131] YSQL, ASH: Create a view to displ…
e834dc7 
…ay wait events information

Summary:
- yb_system_views.sql
-- CREATE VIEW yb_local_tablets AS
--- YB 1a3c113 added the new view yb_wait_event_desc
--- PG 55782d5 removed a lot of views and functions just next to this line
--- Keep yb_wait_event_desc and remove the other views and functions
- yb_pg_rules.out
-- yb_terminated_queries| SELECT d.datname AS databasename,
--- YB 1a3c113 added the test output for the new view yb_wait_event_desc
--- PG 946a8a9 added the comment `-- YB_TODO(jason): update the output for pg15` just after the yb_terminated_queries test output
--- Keep both the lines, but move the comment after the test output of yb_wait_event_desc view

Original summary:
This diff introduces a view `yb_wait_event_desc` to display the
information related to each wait event. The component is not
included in the view because some wait events can be found
in multiple components.

This diff only covers the wait events introduced by YB, the PG
wait events will be covered in a future diff.

This diff also does some minor refactoring
- Deprecate kRpcs_WaitOnMutexInShutdown which was not used anywhere in the code
- Rename YCQLQueryProcessing to YCQLQuery, as it's YSQL counterpart is YSQLQuery

```
yugabyte=# select * from yb_wait_event_desc;
 wait_event_class | wait_event_type |                wait_event                |                                            wait_event_description
------------------+-----------------+------------------------------------------+--------------------------------------------------------------------------------------------------------------
 TServerWait      | Network         | CatalogRead                              | A YSQL backend is waiting for a catalog read from master.
 TServerWait      | Network         | IndexRead                                | A YSQL backend is waiting for a secondary index read from DocDB.
 TServerWait      | Network         | StorageRead                              | A YSQL backend is waiting for a table read from DocDB.
 TServerWait      | Network         | StorageFlush                             | A YSQL backend is waiting for a table/index read/write from DocDB.
 Common           | Cpu             | OnCpu_Active                             | A rpc/task is being actively processed on a thread.
 Common           | Cpu             | OnCpu_Passive                            | A rpc/task is waiting for a thread to pick it up.
 Common           | WaitOnCondition | Idle                                     | The raft log appender/sync thread is idle.
 Common           | WaitOnCondition | Rpc_Done                                 | A rpc is done and waiting for the reactor to send the response to a YSQL/YCQL backend.
 Common           | DiskIO          | RetryableRequests_SaveToDisk             | The in-memory state of the retryable requests is being saved to the disk.                                   +
                  |                 |                                          | This generally happens in the background during a WAL log roll, or Remote bootstrap.
 TabletWait       | WaitOnCondition | MVCC_WaitForSafeTime                     | A read/write rpc is waiting for the SafeTime to be at least the desired read-time.
 TabletWait       | WaitOnCondition | LockedBatchEntry_Lock                    | A read/write rpc is waiting for a DocDB row level lock.
 TabletWait       | WaitOnCondition | BackfillIndex_WaitForAFreeSlot           | A backfill index rpc is waiting for a slot to open if there are too many backfill requests at the same time.
 TabletWait       | DiskIO          | CreatingNewTablet                        | The CreateTablet RPC is creating a new tablet, this may involve writing metadata files, causing I/O wait.
 TabletWait       | DiskIO          | SaveRaftGroupMetadataToDisk              | The raft/tablet metadata is being written to disk. Generally during Snapshot/Restore operations.
 TabletWait       | Network         | TransactionStatusCache_DoGetCommitData   | A rpc needs to look up the commit status of a particular transaction.
 TabletWait       | WaitOnCondition | WaitForYSQLBackendsCatalogVersion        | CREATE INDEX is waiting for YSQL backends to have up-to-date pg_catalog.
 TabletWait       | DiskIO          | WriteSysCatalogSnapshotToDisk            | Writing initial sys catalog snapshot during initdb.
 TabletWait       | WaitOnCondition | DumpRunningRpc_WaitOnReactor             | DumpRunningRpcs is waiting on reactor threads.
 TabletWait       | Network         | ConflictResolution_ResolveConficts       | A read/write rpc is waiting to identify conflicting transactions.
 TabletWait       | WaitOnCondition | ConflictResolution_WaitOnConflictingTxns | A read/write rpc is waiting for conflicting transactions to complete.
 Consensus        | Network         | Raft_WaitingForReplication               | A write rpc is waiting for raft replication.
 Consensus        | Cpu             | Raft_ApplyingEdits                       | A write rpc is applying Raft edits locally.
 Consensus        | DiskIO          | WAL_Append                               | A write rpc is Persisting WAL edits.
 Consensus        | DiskIO          | WAL_Sync                                 | A write rpc is Synchronizing WAL edits.
 Consensus        | DiskIO          | ConsensusMeta_Flush                      | ConsensusMetadata is flushed, say during Raft term/config change or Remote bootstrap etc.
 Consensus        | WaitOnCondition | ReplicaState_TakeUpdateLock              | A write/alter RPC needs to wait for the ReplicaState lock to replicate a batch of writes through Raft.
 RocksDB          | DiskIO          | RocksDB_ReadBlockFromFile                | RocksDB is reading a block from a file.
 RocksDB          | DiskIO          | RocksDB_OpenFile                         | RocksDB is opening a file.
 RocksDB          | DiskIO          | RocksDB_WriteToFile                      | RocksDB is writing to a file.
 RocksDB          | Cpu             | RocksDB_Flush                            | RocksDB is doing a flush.
 RocksDB          | Cpu             | RocksDB_Compaction                       | RocksDB is doing a compaction.
 RocksDB          | WaitOnCondition | RocksDB_PriorityThreadPoolTaskPaused     | RocksDB paused one of flush/compaction tasks for another one with a higher priority
 RocksDB          | DiskIO          | RocksDB_CloseFile                        | RocksDB is closing a file.
 RocksDB          | WaitOnCondition | RocksDB_RateLimiter                      | RocksDB flush/compaction is slowing down due to rate limiter throttling access to disk.
 RocksDB          | WaitOnCondition | RocksDB_WaitForSubcompaction             | RocksDB is waiting for a compaction to complete.
 RocksDB          | DiskIO          | RocksDB_NewIterator                      | RocksDB is waiting for a new iterator to be created
 YCQLQuery        | Cpu             | YCQL_Parse                               | YCQL is parsing a query.
 YCQLQuery        | Cpu             | YCQL_Read                                | YCQL is processing a read query.
 YCQLQuery        | Cpu             | YCQL_Write                               | YCQL is processing a write query.
 YCQLQuery        | Cpu             | YCQL_Analyze                             | YCQL is analyzing a query.
 YCQLQuery        | Cpu             | YCQL_Execute                             | YCQL is executing a query.
 Client           | Network         | YBClient_WaitingOnDocDB                  | YB Client is waiting on DocDB to return a response.
 Client           | Network         | YBClient_LookingUpTablet                 | YB Client is looking up tablet information from the master.
(43 rows)
```
Jira: DB-7929

Test Plan: ./yb_build.sh --java-test TestYsqlUpgrade

Reviewers: jason, tfoucher

Reviewed By: jason

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D37444
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@abhinab-yb abhinab-yb

Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hbhanawat @yugabyte-ci @abhinab-yb