[Feature][Connector][GoogleFirestore-Sink] Support GoogleFirestore Sink #4304

lvshaokang · 2023-03-08T08:22:52Z

Purpose of this pull request

Support google firestore sink.

Check list

Code changed are covered with tests, or it does not need tests for reason:
If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
If you are contributing the connector code, please check that the following files are updated:
1. Update change log that in connector document. For more details you can refer to connector-v2
2. Update plugin-mapping.properties and add new connector information in it
3. Update the pom file of seatunnel-dist
Update the release-note.

TaoZex · 2023-03-13T09:13:06Z

Please provide a screenshot or e2e test to prove that it is available.

lvshaokang · 2023-03-13T09:26:28Z

@TaoZex
I didn't find any available docker image, so I don't know if it is still possible to do e2e testing.
The following is a screenshot of the manual testing.

TaoZex · 2023-03-13T09:32:49Z

LGTM. Thanks for your contribution.

seatunnel-connectors-v2/connector-google-firestore/pom.xml

...ava/org/apache/seatunnel/connectors/seatunnel/google/firestore/sink/FirestoreSinkWriter.java

lvshaokang · 2023-03-15T04:01:14Z

@TyrantLucifer PTAL. Thanks!

liugddx · 2023-03-21T07:09:55Z

you also should update release-note https://github.com/apache/incubator-seatunnel/blob/dev/release-note.md

liugddx · 2023-03-21T07:14:35Z

please add e2e test case. Refer to https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/contribution/coding-guide.md

lvshaokang · 2023-03-21T07:29:58Z

@TaoZex I didn't find any available docker image, so I don't know if it is still possible to do e2e testing. The following is a screenshot of the manual testing.

refer

...e/seatunnel/connectors/seatunnel/google/firestore/exception/FirestoreConnectorErrorCode.java

...2e/connector-google-firestore-e2e/src/test/resources/firestore/fake_to_google_firestore.conf

EricJoy2048 · 2023-04-01T07:14:51Z

I have no problem, @liugddx @TaoZex @TyrantLucifer PTAL again.

...ava/org/apache/seatunnel/connectors/seatunnel/google/firestore/sink/FirestoreSinkWriter.java

…va/org/apache/seatunnel/connectors/seatunnel/google/firestore/sink/FirestoreSinkWriter.java Co-authored-by: TaoZex <[email protected]>

TyrantLucifer · 2023-04-01T09:28:56Z

LGTM, thank you for your contribution.

TaoZex

LGTM

ic4y · 2023-04-21T03:27:48Z

...e2e/src/test/java/org.apache.seatunnel.e2e.connector.google.firestore/GoogleFirestoreIT.java

+        Assertions.assertTrue(documents.size() > 1);
+        Assertions.assertEquals(15, documents.get(0).getData().size());


Refer to src/test/java/org/apache/seatunnel/e2e/connector/starrocks/StarRocksCDCSinkIT.java, it is necessary to check the value of each field in each row

@ic4y PTAL. Thanks

TyrantLucifer

LGTM, cc @hailin0 @EricJoy2048

ic4y · 2023-04-28T04:04:57Z

@lvshaokang Suggest adding the new version documentation for GoogleFirestore Sink in subsequent PR.Or create a related issue

lvshaokang · 2023-04-28T05:34:24Z

@lvshaokang Suggest adding the new version documentation for GoogleFirestore Sink in subsequent PR.Or create a related issue

@ic4y ok, i will supply new doc follow the rules.

lvshaokang added 2 commits March 8, 2023 16:12

[Feature][Connector-V2][GoogleFirestore] Support GoogleFirestore Sink

1844859

Merge branch 'dev' into feat/google-firestore

89a2fb2

TyrantLucifer reviewed Mar 14, 2023

View reviewed changes

unified exception

c845bf0

Merge branch 'dev' into feat/google-firestore

35e11b2

lvshaokang added 2 commits March 22, 2023 15:12

e2e

9f97d06

license

349ae85

EricJoy2048 reviewed Mar 28, 2023

View reviewed changes

...e/seatunnel/connectors/seatunnel/google/firestore/exception/FirestoreConnectorErrorCode.java Show resolved Hide resolved

...2e/connector-google-firestore-e2e/src/test/resources/firestore/fake_to_google_firestore.conf Outdated Show resolved Hide resolved

add docs

548dbff

TaoZex reviewed Apr 1, 2023

View reviewed changes

...ava/org/apache/seatunnel/connectors/seatunnel/google/firestore/sink/FirestoreSinkWriter.java Outdated Show resolved Hide resolved

lvshaokang and others added 2 commits April 1, 2023 16:58

Update seatunnel-connectors-v2/connector-google-firestore/src/main/ja…

c0604eb

…va/org/apache/seatunnel/connectors/seatunnel/google/firestore/sink/FirestoreSinkWriter.java Co-authored-by: TaoZex <[email protected]>

style

4253e13

TyrantLucifer previously approved these changes Apr 1, 2023

View reviewed changes

github-actions bot added approved reviewed labels Apr 1, 2023

TaoZex previously approved these changes Apr 1, 2023

View reviewed changes

Merge branch 'dev' into feat/google-firestore

0ec63e2

ic4y dismissed stale reviews from TaoZex and TyrantLucifer via 0ec63e2 April 20, 2023 13:29

github-actions bot removed approved reviewed labels Apr 20, 2023

ic4y changed the title ~~[Feature][Connector-V2][GoogleFirestore] Support GoogleFirestore Sink~~ [Feature][Connector][GoogleFirestore-Sink] Support GoogleFirestore Sink Apr 21, 2023

ic4y reviewed Apr 21, 2023

View reviewed changes

add data assert

2d5bdbf

TaoZex approved these changes Apr 26, 2023

View reviewed changes

github-actions bot added the reviewed label Apr 26, 2023

TyrantLucifer approved these changes Apr 26, 2023

View reviewed changes

github-actions bot added the approved label Apr 26, 2023

ic4y approved these changes Apr 27, 2023

View reviewed changes

ic4y merged commit f13c261 into apache:dev Apr 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature][Connector][GoogleFirestore-Sink] Support GoogleFirestore Sink #4304

[Feature][Connector][GoogleFirestore-Sink] Support GoogleFirestore Sink #4304

lvshaokang commented Mar 8, 2023

TaoZex commented Mar 13, 2023

lvshaokang commented Mar 13, 2023

TaoZex commented Mar 13, 2023

lvshaokang commented Mar 15, 2023

liugddx commented Mar 21, 2023

liugddx commented Mar 21, 2023

lvshaokang commented Mar 21, 2023

EricJoy2048 commented Apr 1, 2023

TyrantLucifer commented Apr 1, 2023

TaoZex left a comment

ic4y Apr 21, 2023

lvshaokang Apr 22, 2023

TyrantLucifer left a comment

ic4y commented Apr 28, 2023

lvshaokang commented Apr 28, 2023

		Assertions.assertTrue(documents.size() > 1);
		Assertions.assertEquals(15, documents.get(0).getData().size());

[Feature][Connector][GoogleFirestore-Sink] Support GoogleFirestore Sink #4304

[Feature][Connector][GoogleFirestore-Sink] Support GoogleFirestore Sink #4304

Conversation

lvshaokang commented Mar 8, 2023

Purpose of this pull request

Check list

TaoZex commented Mar 13, 2023

lvshaokang commented Mar 13, 2023

TaoZex commented Mar 13, 2023

lvshaokang commented Mar 15, 2023

liugddx commented Mar 21, 2023

liugddx commented Mar 21, 2023

lvshaokang commented Mar 21, 2023

EricJoy2048 commented Apr 1, 2023

TyrantLucifer commented Apr 1, 2023

TaoZex left a comment

Choose a reason for hiding this comment

ic4y Apr 21, 2023

Choose a reason for hiding this comment

lvshaokang Apr 22, 2023

Choose a reason for hiding this comment

TyrantLucifer left a comment

Choose a reason for hiding this comment

ic4y commented Apr 28, 2023

lvshaokang commented Apr 28, 2023