Remove unused table versions along with schema versions in TSchemaVer… #10058

aavdonkin · 2024-10-03T15:00:22Z

…sionNormalizer

Changelog entry

...

Additional information

...

…sionNormalizer

github-actions · 2024-10-03T15:01:44Z

⚪ 2024-10-03 15:01:43 UTC Pre-commit check linux-x86_64-release-asan for 91582ad has started.
⚪ 2024-10-03 15:01:54 UTC Artifacts will be uploaded here
⚪ 2024-10-03 15:04:43 UTC ya make is running...
🟡 2024-10-03 16:26:19 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10221	9869	0	54	262	36

🟢 2024-10-03 16:27:14 UTC Build successful.
🟡 2024-10-03 16:27:43 UTC ydbd size 5.7 GiB changed* by +104.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `9b97c8f`	merge: `91582ad`	diff	diff %
ydbd size	6 086 042 320 Bytes	6 086 149 512 Bytes	+104.7 KiB	+0.002%
ydbd stripped size	1 520 516 016 Bytes	1 520 534 512 Bytes	+18.1 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-10-03T15:01:45Z

⚪ 2024-10-03 15:01:44 UTC Pre-commit check linux-x86_64-relwithdebinfo for 91582ad has started.
⚪ 2024-10-03 15:01:55 UTC Artifacts will be uploaded here
⚪ 2024-10-03 15:04:56 UTC ya make is running...
🟡 2024-10-03 15:51:48 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14987	13508	0	2	1379	98

⚪ 2024-10-03 15:53:05 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-10-03 16:04:34 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
100 (only retried tests)	9	0	0	0	91

🟢 2024-10-03 16:04:41 UTC Build successful.
🟢 2024-10-03 16:05:00 UTC ydbd size 2.8 GiB changed* by +48.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `1faa700`	merge: `91582ad`	diff	diff %
ydbd size	3 001 749 784 Bytes	3 001 799 864 Bytes	+48.9 KiB	+0.002%
ydbd stripped size	477 341 432 Bytes	477 344 568 Bytes	+3.1 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-10-04T07:59:55Z

⚪ 2024-10-04 07:59:55 UTC Pre-commit check linux-x86_64-release-asan for b1dc857 has started.
⚪ 2024-10-04 08:00:05 UTC Artifacts will be uploaded here
⚪ 2024-10-04 08:02:56 UTC ya make is running...
🟡 2024-10-04 09:37:47 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10243	9970	0	101	66	106

🟢 2024-10-04 09:38:40 UTC Build successful.
🟡 2024-10-04 09:39:12 UTC ydbd size 5.7 GiB changed* by +102.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `f595ced`	merge: `b1dc857`	diff	diff %
ydbd size	6 082 037 352 Bytes	6 082 142 640 Bytes	+102.8 KiB	+0.002%
ydbd stripped size	1 519 791 728 Bytes	1 519 810 288 Bytes	+18.1 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-10-04T08:01:46Z

⚪ 2024-10-04 08:01:45 UTC Pre-commit check linux-x86_64-relwithdebinfo for b1dc857 has started.
⚪ 2024-10-04 08:01:57 UTC Artifacts will be uploaded here
⚪ 2024-10-04 08:04:52 UTC ya make is running...
🟡 2024-10-04 08:53:07 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14993	13439	0	25	1425	104

⚪ 2024-10-04 08:54:21 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-10-04 09:05:48 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
171 (only retried tests)	76	0	0	2	93

🟢 2024-10-04 09:05:55 UTC Build successful.
🟢 2024-10-04 09:06:15 UTC ydbd size 2.8 GiB changed* by +49.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `f595ced`	merge: `b1dc857`	diff	diff %
ydbd size	3 001 162 312 Bytes	3 001 213 176 Bytes	+49.7 KiB	+0.002%
ydbd stripped size	477 259 256 Bytes	477 262 456 Bytes	+3.1 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

swalrus1

Некритичные комментарии. Если надо срочно применить нормалайзер, то могу шипнуть.

swalrus1 · 2024-10-08T10:54:31Z

ydb/core/tx/columnshard/normalizer/schema_version/version.cpp

+        for (const auto& id: unusedTableSchemaIds) {
+            if (!maxVersion.has_value() || (id.Version != *maxVersion)) {
+                tablePortion.push_back(id);
+                if (portion.size() + tablePortion.size() >= 10000) {


Конструкция if (... >= 10000) { changes.emplace_back(...); } повторяется -- кажется, это стоит вынести в лямбду и переиспользовать её.

Ещё вектор после std::move() в общем случае невалидный, поэтому, чтобы не было UB, надо его инициализировать заново после мува. По-идее, это может привести к UB в зависимости от реализации STL.

github-actions · 2024-10-08T16:05:42Z

⚪ 2024-10-08 16:05:42 UTC Pre-commit check linux-x86_64-release-asan for 2fa8912 has started.
⚪ 2024-10-08 16:05:53 UTC Artifacts will be uploaded here
⚪ 2024-10-08 16:08:44 UTC ya make is running...
🟡 2024-10-08 17:30:01 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10265	10041	0	67	47	110

🟢 2024-10-08 17:30:58 UTC Build successful.
🟡 2024-10-08 17:31:31 UTC ydbd size 5.7 GiB changed* by +100.0 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `c2a5a57`	merge: `2fa8912`	diff	diff %
ydbd size	6 097 993 744 Bytes	6 098 096 184 Bytes	+100.0 KiB	+0.002%
ydbd stripped size	1 520 325 008 Bytes	1 520 339 472 Bytes	+14.1 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-10-08T16:07:19Z

⚪ 2024-10-08 16:07:19 UTC Pre-commit check linux-x86_64-relwithdebinfo for 2fa8912 has started.
⚪ 2024-10-08 16:07:30 UTC Artifacts will be uploaded here
⚪ 2024-10-08 16:10:24 UTC ya make is running...
🟡 2024-10-08 16:57:33 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14992	13504	0	1	1381	106

⚪ 2024-10-08 16:58:48 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-10-08 17:09:56 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
106 (only retried tests)	5	0	0	0	101

🟢 2024-10-08 17:10:03 UTC Build successful.
🟢 2024-10-08 17:10:23 UTC ydbd size 2.8 GiB changed* by +55.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `c2a5a57`	merge: `2fa8912`	diff	diff %
ydbd size	3 008 717 816 Bytes	3 008 774 888 Bytes	+55.7 KiB	+0.002%
ydbd stripped size	477 379 768 Bytes	477 387 000 Bytes	+7.1 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

ydb-platform#10058)

Remove unused table versions along with schema versions in TSchemaVer…

4104588

…sionNormalizer

aavdonkin requested a review from a team as a code owner October 3, 2024 15:00

github-actions bot added the new-feature label Oct 3, 2024

Added test

1d66f02

swalrus1 requested changes Oct 8, 2024

View reviewed changes

Moved repeated code to lambda

49e9b47

swalrus1 approved these changes Oct 9, 2024

View reviewed changes

aavdonkin merged commit fc0c41d into ydb-platform:main Oct 9, 2024
10 checks passed

This was referenced Oct 21, 2024

Fix kafka cdc read and login without @ #10678

Merged

Allow short auto partitioning strategies yql #10701

Merged

GrigoriyPA mentioned this pull request Oct 23, 2024

YQ-3784 support computation graph invalidation in purecalc #10799

Merged

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 2, 2025

Remove unused table versions along with schema versions in TSchemaVer… (

d238c4e

ydb-platform#10058)

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 5, 2025

Remove unused table versions along with schema versions in TSchemaVer… (

499556c

ydb-platform#10058)

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 5, 2025

Remove unused table versions along with schema versions in TSchemaVer… (

f624da3

ydb-platform#10058)

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 8, 2025

Remove unused table versions along with schema versions in TSchemaVer… (

d77eefb

ydb-platform#10058)

zverevgeny mentioned this pull request Jan 9, 2025

Integrate changes from main #13202

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove unused table versions along with schema versions in TSchemaVer… #10058

Remove unused table versions along with schema versions in TSchemaVer… #10058

aavdonkin commented Oct 3, 2024

github-actions bot commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 4, 2024 •

edited

Loading

github-actions bot commented Oct 4, 2024 •

edited

Loading

swalrus1 left a comment

swalrus1 Oct 8, 2024 •

edited

Loading

github-actions bot commented Oct 8, 2024 •

edited

Loading

github-actions bot commented Oct 8, 2024 •

edited

Loading

Remove unused table versions along with schema versions in TSchemaVer… #10058

Remove unused table versions along with schema versions in TSchemaVer… #10058

Conversation

aavdonkin commented Oct 3, 2024

Changelog entry

Changelog category

Additional information

github-actions bot commented Oct 3, 2024 • edited Loading

github-actions bot commented Oct 3, 2024 • edited Loading

github-actions bot commented Oct 4, 2024 • edited Loading

github-actions bot commented Oct 4, 2024 • edited Loading

swalrus1 left a comment

Choose a reason for hiding this comment

swalrus1 Oct 8, 2024 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Oct 8, 2024 • edited Loading

github-actions bot commented Oct 8, 2024 • edited Loading

github-actions bot commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 4, 2024 •

edited

Loading

github-actions bot commented Oct 4, 2024 •

edited

Loading

swalrus1 Oct 8, 2024 •

edited

Loading

github-actions bot commented Oct 8, 2024 •

edited

Loading

github-actions bot commented Oct 8, 2024 •

edited

Loading