Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix(tianmu):Resolve DDL and insert inot select Possible space inflati…
…on issues(stoneatom#366) Cause of the problem: 1. For multiple versions, Tianmu needs to first copy an original pack when performing DML operations, Modify the copied package and use append write or overwrite write after modification (If there is invalid space in the DATA file that can be written to the current pack, use overwrite write, otherwise use append write) to write to the file, After the latest package is written to a file, the latest version chain will point to the address that was last written. There is a problem with the current (TianmuAttr:: LoadData) logic. Every time you call (TianmuAttr:: LoadData), Will write data to disk, If there are multiple rows written in a transaction, there will be multiple copies of data, "Because the current transaction has not been committed, the space for previous repeated writes has not been released, so the logic of overwriting writes will not be reached.", "I only follow the logic of additional writing, which is the fundamental reason for the skyrocketing space.". If you encounter a particularly large multiline write transaction, it will lead to a space explosion. Moreover, disk IO is performed once per load line, which can also lead to degraded insert performance. Solution: To optimize the logic of (TianmuAttr:: LoadData), it is necessary to determine whether the data in the pack is full before saving changes, Is whether to reach 65536 lines, and if so, write again, If it cannot be reached, it is necessary to write again in the commit phase.
- Loading branch information