Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

examples #24

Merged
merged 14 commits into from
May 23, 2017
Merged

examples #24

merged 14 commits into from
May 23, 2017

Conversation

frairon
Copy link
Contributor

@frairon frairon commented May 23, 2017

No description provided.

@frairon frairon requested a review from flosch May 23, 2017 14:27
### Partitioning and Concurrency

Messages are partitioned in Kafka using the message key. Messages within each partition are processed sequentially,
but in parallel for all partitions. That means for our case, different users can be modified
Copy link
Contributor

@SamiHiltunen SamiHiltunen May 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds a bit wonky, maybe something like Messages are processed sequentially inside a given partition but multiple partitions process messages concurrently.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


Messages are partitioned in Kafka using the message key. Messages within each partition are processed sequentially,
but in parallel for all partitions. That means for our case, different users can be modified
in parallel. A single user however is always modified sequentially, because all messages for that user are assigned
Copy link
Contributor

@SamiHiltunen SamiHiltunen May 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in parallel -> concurrently

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Messages are partitioned in Kafka using the message key. Messages within each partition are processed sequentially,
but in parallel for all partitions. That means for our case, different users can be modified
in parallel. A single user however is always modified sequentially, because all messages for that user are assigned
to the same partition which runs sequentially.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should add a small explanation here as to why the messages for the user are assigned to the same partition, ie. Since we are using the user id as the message's key...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@frairon frairon merged commit 435ddd3 into master May 23, 2017
@frairon frairon deleted the examples branch May 23, 2017 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants