Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bloom ngramms speed up #12982

Merged
merged 11 commits into from
Dec 26, 2024

Conversation

ivanmorozov333
Copy link
Collaborator

  • Performance improvement

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 11:05:13 UTC Pre-commit check linux-x86_64-release-asan for 13813aa has started.
2024-12-25 11:05:24 UTC Artifacts will be uploaded here
2024-12-25 11:08:12 UTC ya make is running...
2024-12-25 11:24:43 UTC Check cancelled

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 11:05:19 UTC Pre-commit check linux-x86_64-relwithdebinfo for 13813aa has started.
2024-12-25 11:05:32 UTC Artifacts will be uploaded here
2024-12-25 11:08:20 UTC ya make is running...
2024-12-25 11:24:41 UTC Check cancelled

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 11:26:26 UTC Pre-commit check linux-x86_64-release-asan for 995931d has started.
2024-12-25 11:28:43 UTC Artifacts will be uploaded here
2024-12-25 11:32:11 UTC ya make is running...
2024-12-25 12:06:43 UTC Check cancelled

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 11:28:29 UTC Pre-commit check linux-x86_64-relwithdebinfo for 995931d has started.
2024-12-25 11:28:41 UTC Artifacts will be uploaded here
2024-12-25 11:31:29 UTC ya make is running...
2024-12-25 12:06:48 UTC Check cancelled

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 12:09:11 UTC Pre-commit check linux-x86_64-relwithdebinfo for 936e746 has started.
2024-12-25 12:09:41 UTC Artifacts will be uploaded here
2024-12-25 12:13:05 UTC ya make is running...
2024-12-25 12:24:18 UTC Check cancelled

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 12:09:34 UTC Pre-commit check linux-x86_64-release-asan for 936e746 has started.
2024-12-25 12:09:46 UTC Artifacts will be uploaded here
2024-12-25 12:12:33 UTC ya make is running...
2024-12-25 12:23:56 UTC Check cancelled

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 12:25:51 UTC Pre-commit check linux-x86_64-relwithdebinfo for c82a98f has started.
2024-12-25 12:27:52 UTC Artifacts will be uploaded here
2024-12-25 12:31:18 UTC ya make is running...
🟡 2024-12-25 13:19:31 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19074 17531 0 2 1418 123

2024-12-25 13:21:39 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-12-25 13:33:35 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
183 (only retried tests) 65 0 0 0 118

🟢 2024-12-25 13:33:43 UTC Build successful.
🟢 2024-12-25 13:34:01 UTC ydbd size 2.1 GiB changed* by +6.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 65567d3 merge: c82a98f diff diff %
ydbd size 2 225 951 840 Bytes 2 225 958 712 Bytes +6.7 KiB +0.000%
ydbd stripped size 467 346 384 Bytes 467 350 480 Bytes +4.0 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 12:27:04 UTC Pre-commit check linux-x86_64-release-asan for c82a98f has started.
2024-12-25 12:27:11 UTC Artifacts will be uploaded here
2024-12-25 12:30:07 UTC ya make is running...
🟡 2024-12-25 13:31:06 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12094 12005 0 25 9 55

2024-12-25 13:32:17 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-12-25 13:39:44 UTC ydbd size 3.6 GiB changed* by +10.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 65567d3 merge: c82a98f diff diff %
ydbd size 3 864 403 440 Bytes 3 864 414 144 Bytes +10.5 KiB +0.000%
ydbd stripped size 1 347 300 880 Bytes 1 347 304 976 Bytes +4.0 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation
2024-12-25 13:39:45 UTC Check cancelled

@ivanmorozov333 ivanmorozov333 marked this pull request as ready for review December 25, 2024 12:47
@ivanmorozov333 ivanmorozov333 requested a review from a team as a code owner December 25, 2024 12:47
Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 13:41:19 UTC Pre-commit check linux-x86_64-release-asan for d6cccb1 has started.
2024-12-25 13:41:38 UTC Check cancelled

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 13:43:04 UTC Pre-commit check linux-x86_64-release-asan for 4a8e344 has started.
2024-12-25 13:43:34 UTC Artifacts will be uploaded here
2024-12-25 13:45:59 UTC Check cancelled

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 13:44:32 UTC Pre-commit check linux-x86_64-relwithdebinfo for 4a8e344 has started.
2024-12-25 13:44:44 UTC Artifacts will be uploaded here
2024-12-25 13:45:59 UTC Check cancelled

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 13:47:45 UTC Pre-commit check linux-x86_64-release-asan for 06cd4cd has started.
2024-12-25 13:48:25 UTC Artifacts will be uploaded here
2024-12-25 13:51:45 UTC ya make is running...
🟡 2024-12-25 15:07:14 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12100 12003 0 22 20 55

2024-12-25 15:08:46 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-25 15:29:26 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
163 (only retried tests) 108 0 4 1 50

2024-12-25 15:29:35 UTC ya make is running... (failed tests rerun, try 3)
🟡 2024-12-25 15:41:54 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
101 (only retried tests) 49 0 4 0 48

🟢 2024-12-25 15:42:01 UTC Build successful.
🟡 2024-12-25 15:42:31 UTC ydbd size 3.6 GiB changed* by +184.5 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 46419b6 merge: 06cd4cd diff diff %
ydbd size 3 864 473 640 Bytes 3 864 662 584 Bytes +184.5 KiB +0.005%
ydbd stripped size 1 347 318 064 Bytes 1 347 383 920 Bytes +64.3 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 13:50:01 UTC Pre-commit check linux-x86_64-relwithdebinfo for 06cd4cd has started.
2024-12-25 13:50:18 UTC Artifacts will be uploaded here
2024-12-25 13:53:06 UTC ya make is running...
🟡 2024-12-25 15:06:08 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19082 17521 0 19 1416 126

2024-12-25 15:08:26 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-25 15:29:14 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
206 (only retried tests) 85 0 2 0 119

2024-12-25 15:29:23 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-12-25 15:41:48 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
176 (only retried tests) 56 0 1 0 119

🟢 2024-12-25 15:41:55 UTC Build successful.
🟢 2024-12-25 15:42:18 UTC ydbd size 2.1 GiB changed* by +6.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 2edc9b5 merge: 06cd4cd diff diff %
ydbd size 2 226 100 904 Bytes 2 226 107 448 Bytes +6.4 KiB +0.000%
ydbd stripped size 467 364 304 Bytes 467 368 400 Bytes +4.0 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@ivanmorozov333 ivanmorozov333 added the rebase-and-check Rebase PR with the current base branch and check label Dec 25, 2024
@github-actions github-actions bot removed the rebase-and-check Rebase PR with the current base branch and check label Dec 25, 2024
Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 18:19:57 UTC Pre-commit check linux-x86_64-relwithdebinfo for 86a4cfa has started.
2024-12-25 18:22:19 UTC Artifacts will be uploaded here
2024-12-25 18:25:08 UTC ya make is running...
🟡 2024-12-25 19:18:25 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19085 17520 0 26 1416 123

2024-12-25 19:20:31 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-25 19:32:22 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
208 (only retried tests) 88 0 1 1 118

2024-12-25 19:32:33 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-12-25 19:44:04 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
172 (only retried tests) 55 0 1 0 116

🟢 2024-12-25 19:44:11 UTC Build successful.
🟢 2024-12-25 19:44:31 UTC ydbd size 2.1 GiB changed* by +6.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: cf03f2f merge: 86a4cfa diff diff %
ydbd size 2 226 280 688 Bytes 2 226 287 248 Bytes +6.4 KiB +0.000%
ydbd stripped size 467 398 288 Bytes 467 402 384 Bytes +4.0 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 25, 2024

2024-12-25 18:20:23 UTC Pre-commit check linux-x86_64-release-asan for 86a4cfa has started.
2024-12-25 18:21:01 UTC Artifacts will be uploaded here
2024-12-25 18:23:50 UTC ya make is running...
🟡 2024-12-25 19:22:58 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11210 11156 0 20 8 26

2024-12-25 19:24:04 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-25 19:32:04 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
88 (only retried tests) 65 0 1 0 22

2024-12-25 19:32:13 UTC ya make is running... (failed tests rerun, try 3)
🟡 2024-12-25 19:36:23 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
46 (only retried tests) 23 0 1 0 22

🟢 2024-12-25 19:36:30 UTC Build successful.
🟢 2024-12-25 19:36:57 UTC ydbd size 3.6 GiB changed* by +14.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: e3d3da9 merge: 86a4cfa diff diff %
ydbd size 3 864 938 648 Bytes 3 864 952 976 Bytes +14.0 KiB +0.000%
ydbd stripped size 1 347 470 576 Bytes 1 347 478 768 Bytes +8.0 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 26, 2024

2024-12-26 08:17:51 UTC Pre-commit check linux-x86_64-relwithdebinfo for b6b67e4 has started.
2024-12-26 08:18:03 UTC Artifacts will be uploaded here
2024-12-26 08:21:07 UTC ya make is running...
🟡 2024-12-26 09:35:25 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
29589 26621 0 6 2838 124

2024-12-26 09:37:59 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-12-26 09:39:12 UTC ydbd size 2.1 GiB changed* by +66.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 9cffaed merge: b6b67e4 diff diff %
ydbd size 2 239 300 192 Bytes 2 239 368 312 Bytes +66.5 KiB +0.003%
ydbd stripped size 471 404 016 Bytes 471 469 168 Bytes +63.6 KiB +0.014%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation
2024-12-26 09:39:14 UTC Check cancelled

Copy link

github-actions bot commented Dec 26, 2024

2024-12-26 08:20:43 UTC Pre-commit check linux-x86_64-release-asan for b6b67e4 has started.
2024-12-26 08:20:55 UTC Artifacts will be uploaded here
2024-12-26 08:23:50 UTC ya make is running...
🟡 2024-12-26 09:39:25 UTC ydbd size 3.6 GiB changed* by +155.5 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 9cffaed merge: b6b67e4 diff diff %
ydbd size 3 865 513 400 Bytes 3 865 672 632 Bytes +155.5 KiB +0.004%
ydbd stripped size 1 347 672 400 Bytes 1 347 851 344 Bytes +174.8 KiB +0.013%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation
2024-12-26 09:39:27 UTC Check cancelled

Copy link

github-actions bot commented Dec 26, 2024

2024-12-26 09:40:04 UTC Pre-commit check linux-x86_64-release-asan for 26e7c13 has started.
2024-12-26 09:40:07 UTC Artifacts will be uploaded here
2024-12-26 09:43:02 UTC ya make is running...
2024-12-26 10:12:16 UTC Check cancelled

Copy link

github-actions bot commented Dec 26, 2024

2024-12-26 09:43:26 UTC Pre-commit check linux-x86_64-relwithdebinfo for 26e7c13 has started.
2024-12-26 09:43:37 UTC Artifacts will be uploaded here
2024-12-26 09:46:31 UTC ya make is running...
2024-12-26 10:12:17 UTC Check cancelled

Copy link

github-actions bot commented Dec 26, 2024

2024-12-26 10:14:30 UTC Pre-commit check linux-x86_64-release-asan for 5bcc9c4 has started.
2024-12-26 10:14:42 UTC Artifacts will be uploaded here
2024-12-26 10:17:37 UTC ya make is running...
2024-12-26 10:17:55 UTC Check cancelled

Copy link

github-actions bot commented Dec 26, 2024

2024-12-26 10:16:50 UTC Pre-commit check linux-x86_64-relwithdebinfo for 5bcc9c4 has started.
2024-12-26 10:17:21 UTC Artifacts will be uploaded here
2024-12-26 10:17:52 UTC Check cancelled

Copy link

github-actions bot commented Dec 26, 2024

2024-12-26 10:19:18 UTC Pre-commit check linux-x86_64-release-asan for 43c238e has started.
2024-12-26 10:19:22 UTC Artifacts will be uploaded here
2024-12-26 10:22:15 UTC ya make is running...
🟡 2024-12-26 11:35:24 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11805 11668 0 63 44 30

2024-12-26 11:36:29 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-26 11:47:42 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
197 (only retried tests) 170 0 2 1 24

2024-12-26 11:47:51 UTC ya make is running... (failed tests rerun, try 3)
🟡 2024-12-26 11:59:27 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
52 (only retried tests) 26 0 2 0 24

🟢 2024-12-26 11:59:34 UTC Build successful.
🔴 2024-12-26 12:00:04 UTC ydbd size 3.6 GiB changed* by +2.1 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 2b752b4 merge: 43c238e diff diff %
ydbd size 3 865 638 872 Bytes 3 867 825 232 Bytes +2.1 MiB +0.057%
ydbd stripped size 1 347 849 936 Bytes 1 348 720 272 Bytes +849.9 KiB +0.065%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 26, 2024

2024-12-26 10:21:57 UTC Pre-commit check linux-x86_64-relwithdebinfo for 43c238e has started.
2024-12-26 10:22:09 UTC Artifacts will be uploaded here
2024-12-26 10:25:15 UTC ya make is running...
🟡 2024-12-26 11:30:09 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
29599 26632 0 4 2836 127

2024-12-26 11:32:40 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-12-26 11:44:16 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
193 (only retried tests) 75 0 0 1 117

🟢 2024-12-26 11:44:23 UTC Build successful.
🟡 2024-12-26 11:44:43 UTC ydbd size 2.1 GiB changed* by +1.3 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 2b752b4 merge: 43c238e diff diff %
ydbd size 2 239 333 088 Bytes 2 240 663 176 Bytes +1.3 MiB +0.059%
ydbd stripped size 471 461 168 Bytes 471 819 952 Bytes +350.4 KiB +0.076%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@ivanmorozov333 ivanmorozov333 merged commit 2f62452 into ydb-platform:main Dec 26, 2024
12 checks passed
@ivanmorozov333 ivanmorozov333 deleted the bloom_ngramms_speedup branch December 26, 2024 12:10
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 2, 2025
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 4, 2025
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 5, 2025
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 5, 2025
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants