feat(gms): add ingestProposalBatch endpoint #10706
Conversation
In my local tests of emitting 1000 MCPs, it was approximately 7x faster to emit MCPs in batches of 100 instead of emitting one at a time. This will need to be an opt-in mechanism in the rest sink until the backend GMS API has been released and stabilized. I have not built that yet.
github-actions
bot
added
ingestion
| return resultUrn.toString(); | ||
|
|
||
| // TODO: We don't actually use this return value anywhere. Maybe we should just stop returning it altogether? | ||
| return "success"; |
There was a problem hiding this comment.
I do think it would be better to return the list of urns that you successfully ingested in the response here to keep in line with what was happening previously, but it is unlikely to cause issues. Just returning "success" does not provide any information beyond what the status code already provides.
There was a problem hiding this comment.
imo returning the urn also doesn't provide much information either - only the status code is important. Ingestion also only looks at the status code, and not the body
I did this for simplicity - I wanted to keep the return type Task<String> for both methods so that the code would be less complex. I can refactor it if you think it's important, but imo we don't gain anything by doing it
In my local test of emitting 1000 MCPs, it was approximately 7x faster to emit MCPs in batches of 100 instead of emitting one at a time.
This will need to be an opt-in mechanism in the rest sink until the backend GMS API has been released and stabilized. I have not built that yet.
Checklist