Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize assignTexters() #26

Merged

Conversation

bchrobot
Copy link
Member

@bchrobot bchrobot commented Feb 7, 2019

Clean up src/workers/jobs.js::assignTexters()

@bchrobot bchrobot changed the base branch from politics-rewired/deploy to main February 7, 2019 19:44
@bchrobot bchrobot changed the base branch from main to politics-rewired/deploy February 7, 2019 19:44
existingAssignments = [],
dynamicAssignments = []
// Do not use `async texter => ...` parallel execution here because `availableContacts`
// needs to be synchronously updated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way to parallelize this would be to replace

for (let index = 0; index < texters.length; index++) {
    // a bunch of asynchronous work
}

with

const textAssignmentPromiseGenerators = texters.map(texter => {
    return () => (async () {
        // a bunch of async stuff here
    }())
})

const chunks = _.chunk(textAssignmentPromiseGenerators, CHUNK_SIZE);
for (let chunk of chunks) {
    await Promise.all(chunk.map(c => c()));
}

Basically what you will have done is created a bunch of functions that, when called, will create a promise with the texter in their closure, and then you get to create all of the stuff in one loop and control the execution nicely

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok wait, you were right about this, since if you run all of the last 5 texters at the same time, they'll all fetch contactsAvailable from the state left at the finish of the previous chunks.lenght - 5, which is not ok!

The way to improve the speed of this if we ever need to is to gather the info up front, compute the amount of texts we'll want to assign to each, and then do what I said in the previous comment

@bchrobot bchrobot changed the title [WIP] Optimize sendMessage: II Optimize assignTexters() Feb 12, 2019
@ben-pr-p ben-pr-p merged commit e4a5b40 into politics-rewired/deploy Feb 12, 2019
@bchrobot bchrobot deleted the politics-rewired/optimize-send-message-ii branch February 15, 2019 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants