Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YSQL][ASH] Circular buffer should be based on the cores available #19134

Closed
1 task done
hbhanawat opened this issue Sep 17, 2023 · 0 comments
Closed
1 task done

[YSQL][ASH] Circular buffer should be based on the cores available #19134

hbhanawat opened this issue Sep 17, 2023 · 0 comments
Assignees
@abhinab-yb
Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@hbhanawat
Copy link
Contributor

hbhanawat commented Sep 17, 2023 •
edited by jira bot
Loading

Jira Link: DB-7932

Description

- We should have different circular buffer sizes and max_events_per_sample for machines with different cores to have better control on memory utilization by AUH

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@hbhanawat hbhanawat added area/ysql Yugabyte SQL (YSQL) status/awaiting-triage Issue awaiting triage area/active-history Active history of sessions labels Sep 17, 2023
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels Sep 17, 2023
@hbhanawat hbhanawat mentioned this issue Sep 17, 2023
Open
55 tasks
@yugabyte-ci yugabyte-ci removed area/active-history Active history of sessions status/awaiting-triage Issue awaiting triage labels Sep 17, 2023
@hbhanawat hbhanawat added the area/active-history Active history of sessions label Sep 18, 2023
@hbhanawat hbhanawat changed the title [YSQL][ActiveHistory] Max number of samples to be collected should be configured based on the memory/cores available [YSQL][ASH] Max number of samples to be collected should be configured based on the memory/cores available Jan 3, 2024
@yugabyte-ci yugabyte-ci removed the area/active-history Active history of sessions label Apr 1, 2024
@yugabyte-ci yugabyte-ci changed the title [YSQL][ASH] Max number of samples to be collected should be configured based on the memory/cores available [YSQL][ASH] Circular buffer should be based on the cores available Sep 3, 2024
abhinab-yb added a commit that referenced this issue Sep 5, 2024
@abhinab-yb
[#19134] YSQL, ASH: Setting ASH circular buffer size based on the num…
a180bef 
…ber of cores

Summary:
The default value of the guc/gflag yb_ash_circular_buffer_size is changed to zero.

If the user doesn't specify the value of guc/gflag, this is overridden based on the
number of cores available on the machine. If the user specifies the value, then
the circular buffer is created with the size the user provides.

The defaults are -
- 32 MiB for 1-2 cores
- 64 MiB for 3-4 cores
- 128 MiB for 5-8 cores
- 256 MiB for 9-16 cores
- 512 MiB for 17-32 cores
- 1024 MiB for > 32 cores
Jira: DB-7932

Test Plan:
Manually tested

In mac
```
$ sysctl -n hw.ncpu
10

$ bin/yb-ctl start --tserver_flags='"allowed_preview_flags_csv=ysql_yb_enable_ash,ysql_yb_ash_enable_infra",ysql_yb_ash_enable_infra=true,ysql_yb_enable_ash=true'

yugabyte=# show yb_ash_circular_buffer_size;
 yb_ash_circular_buffer_size
-----------------------------
 256MB

$ bin/yb-ctl restart --tserver_flags='"allowed_preview_flags_csv=ysql_yb_enable_ash,ysql_yb_ash_enable_infra",ysql_yb_ash_enable_infra=true,ysql_yb_enable_ash=true,ysql_yb_ash_circular_buffer_size=10240'

yugabyte=# show yb_ash_circular_buffer_size;
 yb_ash_circular_buffer_size
-----------------------------
 10MB
```

In alma linux 8

```
$ nproc
4

$ bin/yb-ctl start --tserver_flags='"allowed_preview_flags_csv=ysql_yb_enable_ash,ysql_yb_ash_enable_infra",ysql_yb_ash_enable_infra=true,ysql_yb_enable_ash=true'

yugabyte=# show yb_ash_circular_buffer_size;
 yb_ash_circular_buffer_size
-----------------------------
 64MB

$ bin/yb-ctl restart --tserver_flags='"allowed_preview_flags_csv=ysql_yb_enable_ash,ysql_yb_ash_enable_infra",ysql_yb_ash_enable_infra=true,ysql_yb_enable_ash=true,ysql_yb_ash_circular_buffer_size=10240'

yugabyte=# show yb_ash_circular_buffer_size;
 yb_ash_circular_buffer_size
-----------------------------
 10MB
```

Reviewers: jason, amitanand, hbhanawat

Reviewed By: jason

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D37713
jasonyb pushed a commit that referenced this issue Sep 6, 2024
@yugabyte-ci @jaki
[BACKPORT pg15-cherrypicks] all: Bulk port from master - 94
1463ddd 
Summary:
 c587efd [docs] minor edit (#23796)
 31e09f3 [PLAT-15029] yba installer split data and software directory setup
 f5ba17d [PLAT-15039]: Fix bootstrap on bi-directional xCluster config creation
 578248a [#23770] YSQL: Deterministically populate catalog cache in tests with Connection Manager enabled
 7b1f22a [#23799] test: Fixed PgTableSizeTest.SharedTableSize test for pg15
 788434a [#18771, #21352] docdb: Fix LightweightMessage max size when parsing
 02ced43 [#22821] YSQL: Preserve local limit in a multi-page read
 50ff737 [#23741] docdb: Fix cloning of colocated databases with only parent table
 Excluded: 9889df7 [#23706] YSQL: Add table-level catcache Prometheus metrics
 c770d79 [#23747] MetaCache: Callback should not be called while holding the lock
 Excluded: 40689bc [#22150] YSQL, QueryDiagnostics:  EXPLAIN (ANALYZE, DIST) support for queryDiagnostics
 1655e69 [PLAT-15148]: Set XCluster Table Status to DroppedFromTarget if table in replication is dropped from target only
 16262f7 [#22519] YSQL: Simplify API of the ExplicitRowLockBuffer class
 6614afb [PLAT-14958][PLAT-14959] Make ssh fields optional if skipProvisioning is true
 1153b56 [PLAT-14867] Make sure restart alerts don't trigger for small time updates during NTP sync
 bf1c7bc [PLAT-12226] Add connection pooling status to universe health check
 38d8ae8 [PLAT-14805]Support adding EAR configs
 a180bef [#19134] YSQL, ASH: Setting ASH circular buffer size based on the number of cores
 7d8fc76 Adjust heading link (#23807)
 4c6cf5a [PLAT-4899]Basic validation of certificates
 f24eb10 [#23787] YSQL: Avoid executing conn mgr guc variables hooks for parallel workers
 ee18df8 [PLAT-13921] [K8] [UI] Universe action tasks are disabled after a failed shrink rr node task
 Excluded: dcf1821 [#23797] YSQL: Modify some tests to run in single connection mode with Connection Manager
 0ac22cd [Docs] Remove Drift chat bot (#23802)
 0e91003 [#22825] DocDB: Vector Index General Read Path with DummyANN
 e8f09b5 [PLAT-15175] Make runtime conf for skipping cluster consistency check public
 ee479ee Versionwarning (#23781)
 a05c6a3 [DB-12681] yugabyted-ui: Add Voyager commands to different Voyager phases in the UI.
 cc80d59 [#23777] yugabyted: updating the pg parity testcase to reflect the new gflags enabled for the pg parity feature.

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: jason, tfoucher

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D37822
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@abhinab-yb abhinab-yb

Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hbhanawat @yugabyte-ci @abhinab-yb