Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

COS: Change to official COS object store SDK. #4654

Merged
merged 1 commit into from
Sep 12, 2021

Conversation

hanjm
Copy link
Member

@hanjm hanjm commented Sep 11, 2021

Change to official COS object store SDK.
Fix objstore e2e test invalid test bucket name.
Fix setRange when off>0.

Signed-off-by: hanjm [email protected]

Hi all,
The COS objstore support is implement via github.com/mozillazg/go-cos at 2019 year,
the mozillazg/go-cos is unfficial and not actively update now, it's latest tag version date is 2019-09-18.
the official sdk github.com/tencentyun/cos-go-sdk-v5 which based on mozillazg/go-cos is keeping actively update, the latest tag date is 2021-09-07.

It should be change to official SDK to get more support. I fix testing bucket name bug and setRange bug, then it has passed the e2e objectore test.

thanks to @jojohappy and mozillazg/go-cos author.

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

  1. Change to official COS object store SDK.
  2. Fix objstore e2e test invalid test bucket name.
  3. Fix setRange when off>0.

Verification

run objstore e2e test:

export COS_APP_ID=*** COS_SECRET_ID=*** COS_SECRET_KEY=*** COS_REGION=ap-guangzhou  
$ THANOS_TEST_OBJSTORE_SKIP=AZURE,ALIYUNOSS,BOS,GCS,S3,SWIFT go test ./pkg/block/... ./pkg/compact/... ./pkg/compactv2/... ./pkg/objstore/... ./pkg/shipper/... ./pkg/store/...                       

ok      github.com/thanos-io/thanos/pkg/block   28.246s
ok      github.com/thanos-io/thanos/pkg/block/indexheader       (cached)
ok      github.com/thanos-io/thanos/pkg/block/metadata  (cached)
ok      github.com/thanos-io/thanos/pkg/compact 57.286s
ok      github.com/thanos-io/thanos/pkg/compact/downsample      (cached)
ok      github.com/thanos-io/thanos/pkg/compactv2       (cached)
ok      github.com/thanos-io/thanos/pkg/objstore        (cached)
ok      github.com/thanos-io/thanos/pkg/objstore/azure  (cached)
?       github.com/thanos-io/thanos/pkg/objstore/bos    [no test files]
?       github.com/thanos-io/thanos/pkg/objstore/client [no test files]
ok      github.com/thanos-io/thanos/pkg/objstore/clientutil     (cached)
ok      github.com/thanos-io/thanos/pkg/objstore/cos    0.550s
?       github.com/thanos-io/thanos/pkg/objstore/filesystem     [no test files]
ok      github.com/thanos-io/thanos/pkg/objstore/gcs    (cached)
ok      github.com/thanos-io/thanos/pkg/objstore/objtesting     159.416s
?       github.com/thanos-io/thanos/pkg/objstore/oss    [no test files]
ok      github.com/thanos-io/thanos/pkg/objstore/s3     2.745s
ok      github.com/thanos-io/thanos/pkg/objstore/swift  (cached)
ok      github.com/thanos-io/thanos/pkg/shipper 38.908s
ok      github.com/thanos-io/thanos/pkg/store   197.293s
ok      github.com/thanos-io/thanos/pkg/store/cache     (cached)
?       github.com/thanos-io/thanos/pkg/store/hintspb   [no test files]
ok      github.com/thanos-io/thanos/pkg/store/labelpb   (cached)
ok      github.com/thanos-io/thanos/pkg/store/storepb   (cached)
?       github.com/thanos-io/thanos/pkg/store/storepb/prompb    [no test files]
?       github.com/thanos-io/thanos/pkg/store/storepb/testutil  [no test files]

@hanjm hanjm force-pushed the feature/use-official-cos-sdk branch from ac91ca0 to cb5550d Compare September 11, 2021 09:55
@hanjm
Copy link
Member Author

hanjm commented Sep 11, 2021

How to add auto-tested on CI for COS ? I am glad to add it if possible, like GCS. https://github.com/thanos-io/thanos/blob/main/docs/storage.md#supported-clients. thanks.

@hanjm hanjm force-pushed the feature/use-official-cos-sdk branch from cb5550d to ac11fc9 Compare September 11, 2021 13:27
yeya24
yeya24 previously approved these changes Sep 12, 2021
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pr LGTM.
@hanjm As you are working at Tencent and use COS with Thanos in production, are you willing to help maintain Tencent COS client for Thanos?

cc @jojohappy

Fix objstore e2e test invalid test bucket name.
Fix setRange when off>0.

Signed-off-by: hanjm <[email protected]>
@hanjm
Copy link
Member Author

hanjm commented Sep 12, 2021

Yes, I am willing to help maintain Tencent COS client for Thanos.

@hanjm hanjm requested a review from yeya24 September 12, 2021 09:39
@yeya24 yeya24 enabled auto-merge (squash) September 12, 2021 18:21
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yeya24 yeya24 merged commit 699d31b into thanos-io:main Sep 12, 2021
@yeya24
Copy link
Contributor

yeya24 commented Sep 12, 2021

Yes, I am willing to help maintain Tencent COS client for Thanos.

SGTM. Please open another pr to update the obj store maintainer docs, thank you.

@hanjm hanjm mentioned this pull request Sep 13, 2021
2 tasks
someshkoli pushed a commit to someshkoli/thanos that referenced this pull request Nov 7, 2021
Fix objstore e2e test invalid test bucket name.
Fix setRange when off>0.

Signed-off-by: hanjm <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants