Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'master' into enhance-transaction-functionality
Browse files Browse the repository at this point in the history
# Conflicts:
#	pulsar/transaction_impl.go
reugn committed Dec 16, 2024
2 parents eab85da + 0612938 commit daccfae
Showing 98 changed files with 3,783 additions and 2,350 deletions.
16 changes: 10 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -22,7 +22,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
go-version: [ '1.21', '1.22' ]
go-version: [ '1.22', '1.23' ]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v5
@@ -36,23 +36,27 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-go@v5
with:
go-version: '1.20'
go-version: '1.22'
- name: Check license header
run: docker run --rm -v $(pwd):/github/workspace ghcr.io/korandoru/hawkeye-native:v3 check
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v1.51.2
version: v1.61.0

integration-tests:
runs-on: ubuntu-latest
strategy:
matrix:
go-version: [ '1.21', '1.22' ]
go-version: [ '1.22', '1.23' ]
steps:
- uses: actions/checkout@v3
- name: clean docker cache
run: docker rmi $(docker images -q) -f && df -h
- name: Check for Docker images
id: check_images
run: echo "::set-output name=images::$(docker images -q | wc -l)"
- name: Clean Docker cache if images exist
if: ${{ steps.check_images.outputs.images > 0 }}
run: docker rmi $(docker images -q) -f && df -h
- uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
39 changes: 39 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -23,6 +23,45 @@

All notable changes to this project will be documented in this file.

[0.14.0] 2024-09-23

## Important notice

- The client has dropped support for Go 1.20. The minimum supported version is now Go
1.21. ([Related PR](https://github.com/apache/pulsar-client-go/pull/1249))
- The `BackoffPolicy` interface and related methods has been moved to the `backoff` package from the `internal` package.
The `BackoffPolicy` interface is renamed to
`Policy`. ([Related PR](https://github.com/apache/pulsar-client-go/pull/1197))

## What's Changed
* [improve] Install golang by image in the Dockerfile by @nodece in https://github.com/apache/pulsar-client-go/pull/1246
* [ci] Validate multiple version builds and use golangci-lint-action by @nodece in https://github.com/apache/pulsar-client-go/pull/1250
* [ci] Add merge limitation by @nodece in https://github.com/apache/pulsar-client-go/pull/1248
* [fix] Fix the key-based batch can't guarantee the ordering when flushing by @RobertIndie in https://github.com/apache/pulsar-client-go/pull/1252
* [fix] FailTimeoutMessages cannot delete outdated messages by @nodece in https://github.com/apache/pulsar-client-go/pull/1247
* [fix] Fix pulsar admin revoke subscription permission endpoint by @NoFacePeace in https://github.com/apache/pulsar-client-go/pull/1251
* [fix] Producer close was blocked by @nodece in https://github.com/apache/pulsar-client-go/pull/1249
* [fix] PulsarCtl 1266 Oauth2 Client credentials flow use scopes from the keyfile as well by @Nikolajls in https://github.com/apache/pulsar-client-go/pull/1244
* [feat] Add support for subscription expiration time namespace settings by @klevy-toast in https://github.com/apache/pulsar-client-go/pull/1254
* [fix] Prevent panic when calling Flush on closed producer by @Gilthoniel in https://github.com/apache/pulsar-client-go/pull/1260
* [fix] Avoid a data race when flushing with load by @Gilthoniel in https://github.com/apache/pulsar-client-go/pull/1261
* [improve] Add InitialSubscriptionName for DLQPolicy by @crossoverJie in https://github.com/apache/pulsar-client-go/pull/1264
* [fix] Peek message will return -1 for partitionIndex by @shibd in https://github.com/apache/pulsar-client-go/pull/1267
* [chore] Bump github.com/docker/docker from 27.0.3+incompatible to 27.1.1+incompatible by @dependabot in https://github.com/apache/pulsar-client-go/pull/1269
* [feat] Support the namespace offloadThresholdInSeconds API in pulsaradmin pkg by @ericsyh in https://github.com/apache/pulsar-client-go/pull/1271
* [fix] Stop timer when close timedAckGroupingTracker by @geniusjoe in https://github.com/apache/pulsar-client-go/pull/1279
* [improve] Refactor connection concurrency model by @Gilthoniel in https://github.com/apache/pulsar-client-go/pull/1275
* [fix] Attempt to avoid deadlock during reconnection by @Gilthoniel in https://github.com/apache/pulsar-client-go/pull/1273
* [fix] Fixed panic caused by memory not aligned in arm32 arch by @dream-kzx in https://github.com/apache/pulsar-client-go/pull/1286
* [fix] Reconnection logic and Backoff policy doesn't work correctly by @crossoverJie in https://github.com/apache/pulsar-client-go/pull/1197

## New Contributors
* @NoFacePeace made their first contribution in https://github.com/apache/pulsar-client-go/pull/1251
* @Nikolajls made their first contribution in https://github.com/apache/pulsar-client-go/pull/1244
* @klevy-toast made their first contribution in https://github.com/apache/pulsar-client-go/pull/1254
* @ericsyh made their first contribution in https://github.com/apache/pulsar-client-go/pull/1271
* @dream-kzx made their first contribution in https://github.com/apache/pulsar-client-go/pull/1286

[0.13.1] 2024-08-02

- [fix] Avoid a data race when flushing with load by Gaylor Bosson in [#1261](https://github.com/apache/pulsar-client-go/pull/1261)
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -19,7 +19,7 @@
# set via the Makefile or CLI
ARG PULSAR_IMAGE=apachepulsar/pulsar:latest

ARG GO_VERSION=1.20
ARG GO_VERSION=1.22
FROM golang:$GO_VERSION as golang

FROM $PULSAR_IMAGE
@@ -29,7 +29,7 @@ COPY --from=golang /usr/local/go /pulsar/go

ENV PATH /pulsar/go/bin:$PATH

RUN apt-get update && apt-get install -y git gcc
RUN apk add git gcc musl-dev

### Add pulsar config
COPY integration-tests/certs /pulsar/certs
18 changes: 8 additions & 10 deletions Makefile
Original file line number Diff line number Diff line change
@@ -18,9 +18,9 @@
#

IMAGE_NAME = pulsar-client-go-test:latest
PULSAR_VERSION ?= 3.2.2
PULSAR_VERSION ?= 4.0.0
PULSAR_IMAGE = apachepulsar/pulsar:$(PULSAR_VERSION)
GO_VERSION ?= 1.21
GO_VERSION ?= 1.22
CONTAINER_ARCH ?= $(shell uname -m | sed s/x86_64/amd64/)

# Golang standard bin directory.
@@ -36,7 +36,7 @@ lint: bin/golangci-lint
bin/golangci-lint run

bin/golangci-lint:
GOBIN=$(shell pwd)/bin go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.51.2
GOBIN=$(shell pwd)/bin go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.61.0

# an alternative to above `make lint` command
# use golangCi-lint docker to avoid local golang env issues
@@ -53,27 +53,25 @@ container:
test: container test_standalone test_clustered test_extensible_load_manager

test_standalone: container
docker run -v /var/run/docker.sock:/var/run/docker.sock -i ${IMAGE_NAME} bash -c "cd /pulsar/pulsar-client-go && ./scripts/run-ci.sh"
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -i ${IMAGE_NAME} bash -c "cd /pulsar/pulsar-client-go && ./scripts/run-ci.sh"

test_clustered: container
PULSAR_VERSION=${PULSAR_VERSION} docker compose -f integration-tests/clustered/docker-compose.yml up -d
until curl http://localhost:8080/metrics > /dev/null 2>&1; do sleep 1; done
docker run --network "clustered_pulsar" -i ${IMAGE_NAME} bash -c "cd /pulsar/pulsar-client-go && ./scripts/run-ci-clustered.sh"
docker run --rm --network "clustered_pulsar" -i ${IMAGE_NAME} bash -c "cd /pulsar/pulsar-client-go && ./scripts/run-ci-clustered.sh"
PULSAR_VERSION=${PULSAR_VERSION} docker compose -f integration-tests/clustered/docker-compose.yml down

test_extensible_load_manager: container
PULSAR_VERSION=${PULSAR_VERSION} docker compose -f integration-tests/extensible-load-manager/docker-compose.yml up -d
until curl http://localhost:8080/metrics > /dev/null 2>&1; do sleep 1; done
docker run --network "extensible-load-manager_pulsar" -i ${IMAGE_NAME} bash -c "cd /pulsar/pulsar-client-go && ./scripts/run-ci-extensible-load-manager.sh"
docker run --rm --network "extensible-load-manager_pulsar" -i ${IMAGE_NAME} bash -c "cd /pulsar/pulsar-client-go && ./scripts/run-ci-extensible-load-manager.sh"

PULSAR_VERSION=${PULSAR_VERSION} docker compose -f integration-tests/blue-green/docker-compose.yml up -d
until curl http://localhost:8081/metrics > /dev/null 2>&1 ; do sleep 1; done

# run blue-green migration test (run this test from this env to access both clusters)
go test -race -coverprofile=/tmp/coverage-blue_green_topic_migration -timeout=5m -tags extensible_load_manager -v -run TestBlueGreenMigrationTestSuite ./pulsar
go tool cover -html=/tmp/coverage-blue_green_topic_migration -o coverage-blue_green_topic_migration.html
PULSAR_VERSION=${PULSAR_VERSION} docker compose -f integration-tests/extensible-load-manager/docker-compose.yml down
docker run --rm --network="extensible-load-manager_pulsar" -i ${IMAGE_NAME} bash -c "cd /pulsar/pulsar-client-go && ./scripts/run-ci-blue-green-cluster.sh"
PULSAR_VERSION=${PULSAR_VERSION} docker compose -f integration-tests/blue-green/docker-compose.yml down
PULSAR_VERSION=${PULSAR_VERSION} docker compose -f integration-tests/extensible-load-manager/docker-compose.yml down

clean:
docker rmi --force $(IMAGE_NAME) || true
17 changes: 7 additions & 10 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/apache/pulsar-client-go

go 1.21
go 1.22

require (
github.com/99designs/keyring v1.2.1
@@ -9,7 +9,11 @@ require (
github.com/bits-and-blooms/bitset v1.4.0
github.com/bmizerany/perks v0.0.0-20141205001514-d9a9656a3a4b
github.com/davecgh/go-spew v1.1.1
github.com/golang-jwt/jwt/v5 v5.2.1
github.com/golang/protobuf v1.5.4
github.com/google/uuid v1.6.0
github.com/hamba/avro/v2 v2.26.0
github.com/hashicorp/go-multierror v1.1.1
github.com/klauspost/compress v1.17.9
github.com/onsi/ginkgo v1.16.5
github.com/onsi/gomega v1.19.0
@@ -21,21 +25,14 @@ require (
github.com/spaolacci/murmur3 v1.1.0
github.com/spf13/cobra v1.6.1
github.com/stretchr/testify v1.9.0
github.com/testcontainers/testcontainers-go v0.32.0
go.uber.org/atomic v1.7.0
golang.org/x/mod v0.18.0
golang.org/x/mod v0.20.0
golang.org/x/oauth2 v0.11.0
golang.org/x/time v0.0.0-20220210224613-90d013bbcef8
google.golang.org/protobuf v1.33.0
)

require (
github.com/golang-jwt/jwt/v5 v5.2.1
github.com/golang/protobuf v1.5.4
github.com/hamba/avro/v2 v2.22.2-0.20240625062549-66aad10411d9
github.com/hashicorp/go-multierror v1.1.1
github.com/testcontainers/testcontainers-go v0.32.0
)

require (
dario.cat/mergo v1.0.0 // indirect
github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4 // indirect
8 changes: 4 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
@@ -211,8 +211,8 @@ github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0 h1:YBftPWNWd4WwGqtY2yeZL2ef8rH
github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0/go.mod h1:YN5jB8ie0yfIUg6VvR9Kz84aCaG7AsGZnLjhHbUqwPg=
github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c h1:6rhixN/i8ZofjG1Y75iExal34USq5p+wiN1tpie8IrU=
github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c/go.mod h1:NMPJylDgVpX0MLRlPy15sqSwOFv/U1GZ2m21JhFfek0=
github.com/hamba/avro/v2 v2.22.2-0.20240625062549-66aad10411d9 h1:NEoabXt33PDWK4fXryK4e+XX+fSKDmmu9vg3yb9YI2M=
github.com/hamba/avro/v2 v2.22.2-0.20240625062549-66aad10411d9/go.mod h1:fQVdB2mFZBhPW1D5Abej41LMvrErARGrrdjOnKbm5yw=
github.com/hamba/avro/v2 v2.26.0 h1:IaT5l6W3zh7K67sMrT2+RreJyDTllBGVJm4+Hedk9qE=
github.com/hamba/avro/v2 v2.26.0/go.mod h1:I8glyswHnpED3Nlx2ZdUe+4LJnCOOyiCzLMno9i/Uu0=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
@@ -451,8 +451,8 @@ golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzB
golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.18.0 h1:5+9lSbEzPSdWkH32vYPBwEpX8KwDbM52Ud9xBUvNlb0=
golang.org/x/mod v0.18.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.20.0 h1:utOm6MM3R3dnawAiJgn0y+xvuYRsm1RKM/4giyfDgV0=
golang.org/x/mod v0.20.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
Loading

0 comments on commit daccfae

Please sign in to comment.