Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-43202: [C++][Compute] Detect and explicit error for offset overflow in row table #43226

Merged
merged 5 commits into from
Jul 16, 2024

Conversation

zanmato1984
Copy link
Contributor

@zanmato1984 zanmato1984 commented Jul 11, 2024

Rationale for this change

There are two places for the offset in a row table to possibly overflow: 1) Encoding columns into a row table; 2) Appending to a row table from another row table.

They are particularly bad because they can cause silent wrong result for the computation.

What changes are included in this PR?

Detect such overflows in the aforementioned places and report an explicit error when an overflow is detected.

Are these changes tested?

UT included.

Are there any user-facing changes?

User code that used to be fake passing could see an explicit error after this change.

This PR contains a "Critical Fix".

Copy link

⚠️ GitHub issue #43202 has been automatically assigned in GitHub to PR creator.

@zanmato1984 zanmato1984 added the Critical Fix Bugfixes for security vulnerabilities, crashes, or invalid data. label Jul 11, 2024
@zanmato1984
Copy link
Contributor Author

Hi @pitrou @westonpace , would you like to take a look? Thanks.

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Jul 11, 2024
@zanmato1984
Copy link
Contributor Author

@github-actions crossbow submit -g cpp

This comment was marked as outdated.

@zanmato1984
Copy link
Contributor Author

@github-actions crossbow submit -g cpp

Copy link

Revision: ae34ac4

Submitted crossbow builds: ursacomputing/crossbow @ actions-72e104db5c

Task Status
test-alpine-linux-cpp GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-cuda-cpp GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-fedora-39-cpp GitHub Actions
test-ubuntu-20.04-cpp GitHub Actions
test-ubuntu-20.04-cpp-bundled GitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizer GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions

{
row_encoder.PrepareEncodeSelected(0, num_rows - 1, columns);
ASSERT_OK(row_encoder.EncodeSelected(&row_table, static_cast<uint32_t>(num_rows - 1),
row_ids.data()));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we drop/clear the row_table after this to release a bit of memory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking!

Every time an encoder encodes into a row table, the row table gets effectively cleared. So the memory is only hold til the next EncodeSelected call. Clearing it here doesn't make much difference I guess(?) - the code in between is relatively trivial.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok, thank you

@pitrou pitrou merged commit cf5d7c7 into apache:main Jul 16, 2024
43 checks passed
@pitrou pitrou removed the awaiting committer review Awaiting committer review label Jul 16, 2024
@github-actions github-actions bot added the awaiting committer review Awaiting committer review label Jul 16, 2024
Copy link

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit cf5d7c7.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting committer review Awaiting committer review Component: C++ Critical Fix Bugfixes for security vulnerabilities, crashes, or invalid data.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants