Allow Active Record collection tasks to specify a batch size. #1037
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
adrianna-chang-shopify
force-pushed
the
custom-relation-limit-size
branch
from
d90bccf to
0a9c162
Compare
etiennebarrie
approved these changes
Jun 12, 2024
README.md
Outdated
| @@ -156,6 +156,30 @@ module Maintenance | |||
| end | |||
| ``` | |||
|
|
|||
| #### Customizing the Batch Size | |||
|
|
|||
| The batch size can be customized for tasks that process Active Record Relations. | |||
There was a problem hiding this comment.
In the readme, we mention in the batch collection section that regular active record relations are also batched. But I think we should make it clear here since it's at the top of the readme just below the basics. Also there's some potential confusion with the verb here ("that process Active Record Relations"), since the method is called process and we process records when the collection is a relation.
Maybe something along the lines of:
When processing records from an Active Record Relation, records are fetched in batches internally, before being passed each to the
processmethod.
And then the rest about configuring the batch size.
There was a problem hiding this comment.
Great call out, wording tweaked!
Co-authored-by: Maple Ong <[email protected]>
adrianna-chang-shopify
force-pushed
the
custom-relation-limit-size
branch
from
0a9c162 to
f353578
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes: #890
This PR adds a task DSL for specifying the batch size on an Active Record Relation collection. Currently, job-iteration will default to fetching 100 records with
active_record_on_records, which is rather small for a table with many thousands of rows. Task authors can now customize the batch size per task with acollection_batch_sizemethod.