Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can Multiple Cosmos Trigger Functions on same Cosmos DB Container read different change feeds (documents) for parallel processing? #881

Open
ranjeetkpGit opened this issue Dec 6, 2023 · 3 comments

Comments

@ranjeetkpGit
Copy link

ranjeetkpGit commented Dec 6, 2023

Hi,

Can Multiple Cosmos Trigger Functions (not instances) on same Cosmos DB Container read different change feeds (documents) for parallel processing?

I mean, if there are multiple changed Cosmos documents, they can be processed from multiple functions and each function have new document only not processed document from other function. I understand there is 'Change feed pull model in Azure Cosmos DB'.

@bartelink
Copy link

The more info you can provide about your overall ask, the better people will be able to provide you with guidance; your question is extremely high level.
In general, the principle on which changefeed functionality works is that there is 'a cursor' per physical range.

Each 'processor name' is a cursor (or set of cursors), so one way to get three sets of processing to happen concurrently is to configure three processors (but there will be no coordination across those three sets of work, whic is not what you're after)

Each pull takes a batch of documents from a single physical partition. The work for each normally runs entirely separated. The way the partitions hash mean each assigned processor per range is guaranteed to be working on separate documents.

If you have a 60GB container, you will have at least two partitions. If you have <50 you may not.

@ealsur ealsur transferred this issue from Azure/azure-cosmos-dotnet-v3 Dec 6, 2023
@ealsur
Copy link
Member

ealsur commented Dec 6, 2023

@ranjeetkpGit I am not sure I understand your question.

Are you referring to this: https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/how-to-create-multiple-cosmos-db-triggers which covers how to have multiple independent Azure Functions CosmosDBTriggers configured to listen in the same monitored container?

@ealsur
Copy link
Member

ealsur commented Dec 6, 2023

I transferred the Issue to the correct repository. Issues about Azure Functions Extensions do not belong on the Cosmos DB SDK repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants