Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python examples: please clarify which kinesis connector JAR should be used #42

Open
mattfysh opened this issue Oct 23, 2022 · 1 comment

Comments

@mattfysh
Copy link

mattfysh commented Oct 23, 2022

In this codebase, the JAR listed in several application properties for python examples is one of:

  1. amazon-kinesis-connector-flink-2.0.0.jar
  2. flink-sql-connector-kinesis_2.12-1.13.2.jar
  3. amazon-kinesis-sql-connector-flink-2.0.3.jar
  4. and possibly other variations

For example: https://github.com/aws-samples/amazon-kinesis-data-analytics-examples/blob/master/python/S3Sink/application_properties.json

In the AWS documentation however, the JAR mentioned is flink-sql-connector-kinesis_2.12-1.13.1.jar in places, and also the 1.13.2 version within the same article

Located here: https://docs.aws.amazon.com/kinesisanalytics/latest/java/gs-python-createapp.html

Please clarify which of these dependencies should be included in the uploaded ZIP file

@astamur
Copy link

astamur commented Nov 1, 2022

Hello!

  • flink-sql-connector-kinesis_2.12-1.13.x.jar: use this connector if you're planning to deploy it as a Flink 1.13 application (AWS documentation is correct)
  • amazon-kinesis-connector-flink connector is used for Flink 1.8 and 1.11 for DataStreamAPI, but python apps use TableAPI
  • amazon-kinesis-sql-connector is used for TableAPI in Flink 1.8 and 1.11
    This versions should be used on today:
Flink 1.8: Amazon 1.6.1
Flink 1.11: Amazon 2.4.1
Flink 1.13: Apache >=1.13.3
Flink 1.15: Apache 1.15.2

Our python code examples still use a connector for Flink 1.11. We're planning to update examples soon together with Flink 1.15 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants