Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: klip-41: ksqlDB .Net Client #6613

Merged
merged 4 commits into from
Nov 24, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion design-proposals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,5 +81,5 @@ Next KLIP number: **43**
| KLIP-37: System Tables | Proposal | | | |
| [KLIP-38: Variable Substitution](klip-38-variable-substitution.md) | Merged | 0.14.0 | 6.1.0 | [Discussion](https://github.com/confluentinc/ksql/pull/6259) |
| KLIP-40: Prepared Statements for Java Client | Proposal | | | |
| KLIP-41: ksqlDB .NET Client | Proposal | | | |
| [KLIP-41: ksqlDB .NET Client](klip-41-ksqldb-.net-client.md) | Proposal | | | [Discussion](https://github.com/confluentinc/ksql/pull/6613)|
| KLIP-42: Schema Migrations Tool | Proposal | | | |
72 changes: 72 additions & 0 deletions design-proposals/klip-41-ksqldb-.net-client.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# KLIP 41 - ksqlDB .NET Client

**Author**: Alex Basiuk (@alex-basiuk) |
**Release Target**: TBD |
**Status**: _In Discussion_ |
**Discussion**: TBD

**tl;dr:** _Create a ksqlDB client for .NET which is on par with the existing Java client.
The .NET community will benefit from the this project. The goal is to provide first class developer experience and
simplify writing event streaming applications in .NET._

## Motivation and background

Basically, it's similar to [KLIP 15](./klip-15-new-api-and-client.md).
It should lower the barrier for .NET developers and increase adoption of ksqlDB by the community.
The library should offer the same feature set as the Java library and should follow common .NET conventions and patterns.

## What is in scope

* A new client implemented in C#.
* Initially, the client will be targeted at .NET Core 3 and .NET Framework 5. Support of older versions will be added later.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not versed in .NET, but one thing I'm wondering is how support for multiple versions will look over time. Any ideas about how to manage that workflow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MichaelDrogalis .NET 5 and .NET Core 3 are very close in terms of APIs. I don't think .NET 5 has any breaking changes in the libraries required by this project. It won't be hard to support both until end of support of .NET Core 3.1. Basically, you can have one project which targets multiple framework versions.
Support of older versions is probably more involved.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, that makes sense. Mostly just asking since Im unfamiliar with the .NET stack.

* The client will support execution and streaming of queries (both push and pull), inserting of rows into streams, DDL operations and admin operations
such as list and describe.
* The client will utilise C# [async streams](https://docs.microsoft.com/en-us/dotnet/csharp/tutorials/generate-consume-asynchronous-stream) feature.
* The client will also support a direct / synchronous interaction style.
* High-quality documentation and examples.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super. :) If this is going to live inside of a repo owned by you for a while, we can put the documentation in a README there so it is self-contained.


## What is not in scope

No plans to provide more functional API for F#. However, it will be possible to use the client in F#.

## Value/Return

Ideas of event sourcing / CQRS / DDD style apps are popular in the .NET community but ksqlDB is often perceived as a Java-specific technology.
A native client will make it easier to try and hopefully adopt ksqlDB by .NET developers.

## Public APIS

N/A

## Design

* The client will target multiple [target frameworks](https://docs.microsoft.com/en-us/dotnet/standard/frameworks).
Initially it will be netcoreapp3.0 and net5.0. Support of older versions e.g. netstandard2.0 will be added at later stage.
* The client will provide an API based on async streams for streaming and async methods for the rest.
* The client will support injectable [Polly policies](https://github.com/App-vNext/Polly) to enable users to use custom resilience policies. However, out of the box, it should use sensible default retry policies.
* The client will expose convenience builder extension methods for ASP.NET Core. It should be easy to add and configure the client in ASP.NET Core idiomatic way.
But the mechanism will be optional. Users won't be required to use it or another DI.
* The client will have minimal dependencies on third party libraries.
* The client will follow performance best practices.
* The client will be thread-safe.

## Test plan

Unit and integration test will be mandatory. Performance benchmarks are desirable but unlikely be the first priority.
Ideally, tests should be executed on both Windows and Linux platforms.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do any CI providers make dual Windows tests easy? Been a while since I've tested much on Windows.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MichaelDrogalis Travis, CircleCI, Azure CI. Folks from confluent-kafka-dotnet use Travis.


## LOEs and Delivery Milestones
TBD

## Documentation Updates

* High quality documentation is required.
* Example applications showing how to use the client in a real app.

## Compatibility Implications

N/A

## Security Implications

The client will support all protocols supported by the ksqlDB REST API.