Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tianmu): merge to Stonedb 5.7 stable #1915

Merged
merged 124 commits into from
Jun 29, 2023
Merged

feat(tianmu): merge to Stonedb 5.7 stable #1915

merged 124 commits into from
Jun 29, 2023

Conversation

RingsC
Copy link
Contributor

@RingsC RingsC commented Jun 29, 2023

Summary about this PR

Issue Number: close #issue_number_you_created

Tests Check List

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Changelog

  • New Feature
  • Bug Fix
  • Performance Improvement
  • Build/Testing/CI/CD
  • Documentation
  • Not for changelog (changelog entry is not required)

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features

Nliver and others added 30 commits March 21, 2023 14:07
…anmu_no_key_error (#1462)"

In version 1.0.4, we will discard "MANDATORY_TIANMU" and "NO_KEY_ERROR" in (sql_mode)

Specifies whether to enable mandatory Tianmu engine in table. if yes ,set tianmu_mandatory to ON, otherwise set the variable to OFF.
Specifies whether to to directly skip DDL statements that are not supported by the SQL layer, instead of reporting errors.
if yes ,set tianmu_no_key_error to ON, otherwise set the variable to OFF.
…primary/secondary synchronization if UUIDs are used as the primary key(#1464)

Cause of the problem:
When performing a primary key scan under the master slave, "ha_tianmu:: position()" is called first to obtain the primary key value from the "record",
However, in this scenario, after calling the "key_copy()" function, the "record" will be cleared, resulting in the subsequent "GetKeys()" obtaining a null primary key value.
Solution:
Because the value "handler->ref" is not used in the future, you can simply delete the call to "key_copy()".
… testcases, add date type and std func testcase(#1196)
    1. Fix the crash first
    2. then redesign the entire aggregated data stream
Bumps [nth-check](https://github.com/fb55/nth-check) to 2.1.1 and updates ancestor dependency [unist-util-select](https://github.com/syntax-tree/unist-util-select). These dependencies need to be updated together.


Updates `nth-check` from 1.0.2 to 2.1.1
- [Release notes](https://github.com/fb55/nth-check/releases)
- [Commits](fb55/nth-check@v1.0.2...v2.1.1)

Updates `unist-util-select` from 2.0.2 to 4.0.1
- [Release notes](https://github.com/syntax-tree/unist-util-select/releases)
- [Commits](syntax-tree/unist-util-select@2.0.2...4.0.1)

---
updated-dependencies:
- dependency-name: nth-check
  dependency-type: indirect
- dependency-name: unist-util-select
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
update the Sponsor button
fix the link of opencollective
…on issues(#366)

Cause of the problem:
1. For multiple versions, Tianmu needs to first copy an original pack when performing DML operations,
Modify the copied package and use append write or overwrite write after modification
(If there is invalid space in the DATA file that can be written to the current pack, use overwrite write, otherwise use append write) to write to the file,
After the latest package is written to a file, the latest version chain will point to the address that was last written.
There is a problem with the current (TianmuAttr:: LoadData) logic. Every time you call (TianmuAttr:: LoadData),
Will write data to disk,
If there are multiple rows written in a transaction, there will be multiple copies of data,
"Because the current transaction has not been committed, the space for previous repeated writes has not been released, so the logic of overwriting writes will not be reached.",
"I only follow the logic of additional writing, which is the fundamental reason for the skyrocketing space.".
If you encounter a particularly large multiline write transaction, it will lead to a space explosion.
Moreover, disk IO is performed once per load line, which can also lead to degraded insert performance.
Solution:
To optimize the logic of (TianmuAttr:: LoadData), it is necessary to determine whether the data in the pack is full before saving changes,
Is whether to reach 65536 lines, and if so, write again,
If it cannot be reached, it is necessary to write again in the commit phase.
)

Currently, TIANMU does not support manual transactions and only supports automatic commit.
However, it does determine whether to commit the transaction based on the automatic commit parameters of MySQL.
If automatic commit is turned off, automatic commit will not be performed on the transaction
RingsC and others added 20 commits June 8, 2023 12:40
1: Fixup the memory leakage of aggregation function, which may lead to failed of malloc.
2: Re-impl the operator= of `ValueOrNull`.
3: Fixup the assertion of `dow_cast` in ` Query::ClearSubselectTransformation`, `Item_func_trig_cond`.
[summary]
case_when.test
drop_restric.test
empty_string_not_null.test
left_right_func.test
like_not_like.test
multi_join.test
order_by.test
ssb_small.test
union_case.test
To support `update ignore` statement. The logic of uniqueness check is re-implemented.
Cause:
  in the function ParsingStrategy::ParseResult ParsingStrategy::GetOneRow
field->val_str(str) cannot distinguish 0 and NULL value.
Solution:
  Check whether field's default value is NULL.
…m clause

1: To fixup unsupport union or union all a sql statement which is without from clause.
2: Re-format some codes and functions.
1:Removes the unnessary optimization in stage of compiliation of tianmu. It doesnot have any helps for us.
and may introuduce unexepected behaviors.
2:Refine MTR: issue848, issue1865, alter_table1, issue1523
In multi-thread aggregation, ExpressionColumn will occur double free due to without
protection. Thread A will do ValueOrNull::operator ==, but in thread B, it will try to
free it. Therefore, it leads to instance crash.
…_max #1564

[summary]
1. static_cast<int64_t>(18446744073709551601) = -15
2. Item will set 18446744073709551601 with unsigned flag, but in tianmu
transform to ValueOrNot, the value will be set to `-15`.
3. add `unsigned flag` in value_or_null & TianmuNum & tianmu expr.
…ision loss problem (#1173)

When converting TIME/DATETIME to ulonglong numeric, tianmu engine does not take the
TIME_to_ulonglong_time_round process. This causes the results different from innodb.
Furthermore, when we close the tianmu_insert_delayed parameter and execute an insert SQL,
the TIME/DATETIME/TIMESTAMP type's data will loss precision due to incomplete attribute copying.

PR Close #1173
files deleted:
        storage/tianmu/core/rc_attr_typeinfo.h
        storage/tianmu/handler/tianmu_handler.cpp
        storage/tianmu/handler/tianmu_handler_com.cpp
        storage/tianmu/types/rc_data_types.cpp
        storage/tianmu/types/rc_num.cpp
        storage/tianmu/types/rc_num.h
        storage/tianmu/types/rc_value_object.cpp
@mergify
Copy link
Contributor

mergify bot commented Jun 29, 2023

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

@mergify mergify bot added the PR-feature feature for pull request label Jun 29, 2023
@RingsC RingsC marked this pull request as ready for review June 29, 2023 09:40
@RingsC RingsC requested review from Nliver and Xinqiangxu June 29, 2023 11:16
Copy link
Collaborator

@Nliver Nliver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov
Copy link

codecov bot commented Jun 29, 2023

Codecov Report

Patch coverage: 68.30% and project coverage change: +17.28 🎉

Comparison is base (b6380d1) 42.90% compared to head (422bd73) 60.19%.

Additional details and impacted files
@@                   Coverage Diff                   @@
##           stonedb-5.7-stable    #1915       +/-   ##
=======================================================
+ Coverage               42.90%   60.19%   +17.28%     
=======================================================
  Files                    1838     1837        -1     
  Lines                  397387   380689    -16698     
=======================================================
+ Hits                   170509   229148    +58639     
+ Misses                 226878   151541    -75337     
Impacted Files Coverage Δ
sql/item.h 51.43% <ø> (+10.31%) ⬆️
sql/sql_class.h 79.34% <ø> (+8.40%) ⬆️
sql/sql_load.cc 87.36% <ø> (+80.98%) ⬆️
sql/sql_show.cc 84.70% <ø> (+34.99%) ⬆️
sql/sql_union.cc 89.54% <ø> (+14.37%) ⬆️
sql/sys_vars.cc 69.93% <ø> (+53.30%) ⬆️
storage/tianmu/common/common_definitions.h 73.68% <ø> (-10.53%) ⬇️
storage/tianmu/compress/basic_data_filt.h 10.00% <ø> (ø)
storage/tianmu/compress/dictionary.cpp 60.55% <ø> (ø)
storage/tianmu/compress/num_compressor.h 48.11% <ø> (-2.61%) ⬇️
... and 121 more

... and 1169 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@RingsC RingsC merged commit a43394b into stoneatom:stonedb-5.7-stable Jun 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR-feature feature for pull request
Projects
None yet
Development

Successfully merging this pull request may close these issues.