feat: undolog add decompressor func #457

wang1309 · 2023-01-15T03:13:38Z

What this PR does:
add undo log decompressor func

Which issue(s) this PR fixes:
none

Fixes #449

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Init

init github actions

Init

fix cli

add apache license

codecov-commenter · 2023-01-15T03:42:38Z

Codecov Report

Merging #457 (b244cab) into master (437f712) will decrease coverage by 0.72%.
The diff coverage is 0.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##           master     #457      +/-   ##
==========================================
- Coverage   39.05%   38.33%   -0.72%     
==========================================
  Files         144      150       +6     
  Lines        9120    10011     +891     
==========================================
+ Hits         3562     3838     +276     
- Misses       5264     5855     +591     
- Partials      294      318      +24

Impacted Files	Coverage Δ
pkg/client/client.go	`0.00% <ø> (ø)`
pkg/compressor/compressor_factory.go	`0.00% <0.00%> (ø)`
pkg/compressor/compressor_type.go	`0.00% <0.00%> (ø)`
pkg/compressor/none_compressor.go	`0.00% <0.00%> (ø)`
pkg/datasource/sql/tx_at.go	`23.52% <0.00%> (-2.19%)`	⬇️
pkg/datasource/sql/parser/parser_factory.go	`50.00% <0.00%> (-0.95%)`	⬇️
pkg/datasource/sql/exec/at/delete_executor.go	`23.23% <0.00%> (-0.24%)`	⬇️
pkg/rm/init.go	`0.00% <0.00%> (ø)`
pkg/client/config.go	`61.66% <0.00%> (ø)`
... and 10 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

pkg/compressor/compressor_factory.go

pkg/compressor/compressor_type.go

georgehao · 2023-01-17T03:19:58Z

pkg/compressor/compressortype_string.go

+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with


这个文件不要

这个暂时好像不能删，项目编译的时候没有执行go generate，导致CI通过不了。

这个暂时好像不能删，项目编译的时候没有执行go generate，导致CI通过不了。

这个文件的作用是来干嘛的呀

用于获取常量的字符串

用于获取常量的字符串

为啥会影响CI呢？看看报啥错，挺奇怪

这个文件看看可以删除了吗

luky116 · 2023-02-07T00:00:19Z

pkg/compressor/deflate_compress.go


 	"github.com/seata/seata-go/pkg/util/log"
 )

 type DeflateCompress struct{}

+var (
+	deflateOnce             sync.Once


所有的单例都去了吧，每次返回一个新对象

pkg/compressor/compressor_factory.go

luky116 · 2023-02-07T00:22:39Z

pkg/compressor/compressor.go

@@ -17,21 +17,6 @@

 package compressor

-type CompressorType int8


这个看看是否可以直接写在这里

感觉可能还是放在compress_type 文件一起好点，比较内聚一点。

luky116 · 2023-02-07T00:29:21Z

pkg/constant/client_table_columns_name.go

+	// UndoLogId The constant undo_log column name xid
+	// this field is not use in mysql
+	UndoLogId = "id"
+


这些变量如果没用到，就删了吧

luky116 · 2023-02-07T00:31:25Z

pkg/datasource/sql/undo/base/undo.go

@@ -213,7 +215,7 @@ func (m *BaseUndoLogManager) FlushUndoLog(tranCtx *types.TransactionContext, con

 	parseContext := make(map[string]string, 0)
 	parseContext[serializerKey] = "jackson"
-	parseContext[compressorTypeKey] = "NONE"
+	parseContext[compressorTypeKey] = compressor.CompressorNone.String()


这里从配置文件读取使用了哪个压缩方式

看代码seata-go这里默认是不压缩，应该没做这一步。

luky116 · 2023-02-07T00:39:36Z

pkg/datasource/sql/undo/base/undo.go

@@ -480,16 +494,30 @@ func (m *BaseUndoLogManager) DecodeMap(str string) map[string]string {
 	return res
 }

+func (m *BaseUndoLogManager) UnmarshalContext(undoContext []byte) (map[string]string, error) {
+	res := make(map[string]string)


这里是和这个方法做对对应吗？看这里的逻辑有点复杂，再确认下哈

存储的时候用的 json 格式

所以这里用 unmarshal 是和存储一一对应的，应该没啥问题。

106umao · 2023-02-12T01:33:27Z

testdata/undo_test.go

+	testUndoLog := func() {
+		manager := mysql.NewUndoLogManager()
+
+		db, err := sql.Open(sql2.SeataATMySQLDriver, "root:12345678@tcp(127.0.0.1:3306)/seata_client?multiStatements=true")


数据库操作应该是可以 Mock 的，另外这个测试文件放在 pkg/datasource/sql/undo/base 下比较合适

放在这里会有循环引用问题，这个文件也可以先删了

106umao · 2023-02-12T01:34:26Z

testdata/dockercompose/mysql/mysqld.cnf

+# By default we only accept connections from localhost
+bind-address   =  0.0.0.0
+# Disabling symbolic-links is recommended to prevent assorted security risks
+symbolic-links=0


缺少换行符

106umao · 2023-02-12T01:35:33Z

testdata/dockercompose/docker-compose.yml

+      - ./mysql:/docker-entrypoint-initdb.d
+      - ./mysql/mysqld.cnf:/etc/mysql/mysql.conf.d/mysqld.cnf
+    ports:
+      - "3306:3306"


这个文件应该拆一个 pr

应该误提交了，我回退一下

106umao · 2023-02-12T01:35:42Z

testdata/dockercompose/docker-health-check.sh

+sleep 5
+curl http://127.0.0.1:7091
+sleep 10
+


106umao · 2023-02-12T01:36:42Z

testdata/dockercompose/mysql/order.sql

+                            `ext` varchar(100) DEFAULT NULL,
+                            PRIMARY KEY (`id`),
+                            KEY `idx_unionkey` (`xid`,`branch_id`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8;


luky116 · 2023-02-11T11:13:55Z

pkg/compressor/compressortype_string.go

+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with


用于获取常量的字符串

为啥会影响CI呢？看看报啥错，挺奇怪

luky116 · 2023-02-11T13:05:31Z

pkg/compressor/compressor_type.go

+var compressor map[string]CompressorType
+
+func GetByName(name string) CompressorType {
+	if compressor == nil {


改为switch...case，每次返回一个新的对象，这样就不要用这个map了

嗯嗯，请教一下为何改为 switch case,感觉这里实现问题不大。

luky116 · 2023-02-25T09:37:38Z

pkg/compressor/zstd_compress.go

@@ -23,6 +23,10 @@ import (

 type Zstd struct{}

+func NewZstd() *Zstd {


如果new没有参数，可以不用写New函数？

嗯，感觉还是有好一点，这能够收敛一下，后期也好维护。

luky116 · 2023-02-25T09:42:45Z

pkg/compressor/compressortype_string.go

+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with


这个文件看看可以删除了吗

luky116 · 2023-02-25T09:44:49Z

pkg/datasource/sql/undo/base/undo.go

@@ -354,7 +368,7 @@ func (m *BaseUndoLogManager) insertUndoLogWithGlobalFinished(ctx context.Context
 	// todo use config to replace
 	parseContext := make(map[string]string, 0)
 	parseContext[serializerKey] = "jackson"
-	parseContext[compressorTypeKey] = "NONE"
+	parseContext[compressorTypeKey] = compressor.CompressorNone.String()


这里能读取下配置文件吗

嗯，这块配置现在有了吗，我看有个todo

luky116 · 2023-02-25T09:51:54Z

pkg/compressor/compressor_type.go

+	CompressorSevenz
+	CompressorBzip2
+	CompressorLz4
+	CompressorDefault


CompressorDefault和CompressorMaxl好像和java对不上？
CompressorDeflate应该在CompressorZstd前面

luky116 · 2023-02-25T12:37:31Z

pkg/compressor/compressor_factory.go

+var compressorFactory map[string]Compressor
+
+func (c CompressorType) GetCompressor() Compressor {
+	if compressorFactory == nil {


switch

不用单例模式

luky116 · 2023-02-25T12:38:51Z

pkg/compressor/bzip2_compress.go

@@ -27,6 +27,10 @@ import (
 type Bzip2 struct {
 }

+func NewBizp2() *Bzip2 {


没有参数不用New

luky116 · 2023-02-25T12:42:23Z

pkg/compressor/compressor_type.go

+type CompressorType int8
+
+const (
+	CompressorNone CompressorType = iota


加一个单测

luky116 · 2023-02-25T12:47:30Z

pkg/compressor/compressortype_string.go

+
+var _CompressorType_index = [...]uint8{0, 14, 28, 41, 57, 72, 85, 102, 116, 133, 147}
+
+func (i CompressorType) String() string {


手动写String函数

luky116 · 2023-02-25T12:50:32Z

pkg/datasource/sql/undo/base/undo.go

@@ -213,7 +215,7 @@ func (m *BaseUndoLogManager) FlushUndoLog(tranCtx *types.TransactionContext, con

 	parseContext := make(map[string]string, 0)
 	parseContext[serializerKey] = "jackson"
-	parseContext[compressorTypeKey] = "NONE"
+	parseContext[compressorTypeKey] = compressor.CompressorNone.String()


AlexStocks and others added 30 commits May 3, 2022 09:30

Initial commit

3a50c2a

add config file

4c6ad0f

init

004badf

add rm

f646586

add processor

580ab4a

bug fix

0939057

use processor on message

b6e759a

change default config

4016001

Merge pull request apache#1 from luky116/init

4579084

Init

init github Actions

9f2665a

init github Actions

3fd0b4c

Merge pull request apache#2 from liiibpm/main

1944ae1

init github actions

add tcc process

9095355

Adjust the structure of the project

dd38d07

Adjust the structure of the project

634657b

Adjust the structure of the project

6d0ea72

fix linter error

83cccac

Merge pull request apache#4 from luky116/init

bfa69cc

Init

fix cli

7ceb462

Merge pull request apache#5 from luky116/init

8a78ef8

fix cli

add apache license

986e5f8

Merge pull request #6 from luky116/init

38b6804

add apache license

fix config

4c36bc4

add unit test

114147c

add go imports shell

ed9b0a7

Adjust project directory

a1da44e

remove iota

1044c64

fix cli bug

29d735b

remove the useless comments

681f298

Merge remote-tracking branch 'my/addUnitTest' into addUnitTest

7511240

wang1309 added 3 commits January 15, 2023 11:18

fix: ci fail fix

e5e21b2

fix: ci fail fix

2f3bfc4

fix: ci fail fix

9404a61

georgehao reviewed Jan 17, 2023

View reviewed changes

pkg/compressor/compressor_factory.go Outdated Show resolved Hide resolved

georgehao reviewed Jan 17, 2023

View reviewed changes

pkg/compressor/compressor_type.go Outdated Show resolved Hide resolved

georgehao reviewed Jan 17, 2023

View reviewed changes

wang1309 added 2 commits February 4, 2023 09:45

fix:delete init func && compressortype_string

4b0aff5

fix:add compressor_type_string

2fcb612

luky116 reviewed Feb 7, 2023

View reviewed changes

fix:optimize code

d6554f1

106umao changed the title ~~Feat undolog add decompressor func~~ feat undolog add decompressor func Feb 11, 2023

106umao changed the title ~~feat undolog add decompressor func~~ feat: undolog add decompressor func Feb 11, 2023

106umao suggested changes Feb 12, 2023

View reviewed changes

wang1309 added 2 commits February 16, 2023 21:20

fix code review porblem

cfdea8b

Merge branch 'master' into feat_undolog_addComposer

b89174c

luky116 reviewed Feb 25, 2023

View reviewed changes

wang1309 added 3 commits February 26, 2023 09:59

fix:code review fixed

b244cab

Merge remote-tracking branch 'upstream/master'

77130c7

Merge branch 'master' into feat_undolog_addComposer

12ea3b7

github-actions bot removed the module/rm label Mar 9, 2023

wang1309 added 2 commits March 9, 2023 21:14

fix:solve conflict

dc9e442

fixed:solve conflict

42c5079

106umao approved these changes Mar 14, 2023

View reviewed changes

luky116 approved these changes Mar 14, 2023

View reviewed changes

georgehao approved these changes Mar 14, 2023

View reviewed changes

georgehao merged commit 46da59f into apache:master Mar 14, 2023

georgehao pushed a commit to georgehao/seata-go that referenced this pull request May 7, 2023

feat: undolog add decompressor func (apache#457)

1607833

		@@ -17,21 +17,6 @@

		package compressor

		type CompressorType int8

		@@ -23,6 +23,10 @@ import (

		type Zstd struct{}

		func NewZstd() *Zstd {


		var _CompressorType_index = [...]uint8{0, 14, 28, 41, 57, 72, 85, 102, 116, 133, 147}

		func (i CompressorType) String() string {

feat: undolog add decompressor func #457

feat: undolog add decompressor func #457

Conversation

wang1309 commented Jan 15, 2023

codecov-commenter commented Jan 15, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luky116 Feb 11, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Jan 15, 2023 •

edited

Loading

luky116 Feb 11, 2023 •

edited

Loading