Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.x] Add takeUntilTimeout method to the lazy collection #34444

Merged
merged 3 commits into from
Sep 22, 2020

Conversation

JosephSilber
Copy link
Contributor

@JosephSilber JosephSilber commented Sep 21, 2020

This PR adds a takeUntilTimeout to the LazyCollection class, which lets you stop a long running process after a given timeout:

$lazyCollection
    ->takeUntilTimeout(now()->add(2, 'minutes'))
    ->each(fn ($item) => doSomethingThatMayTakeSomeTime($item));

// ^^ This will only process items for up to 2 minutes ^^

This is especially useful for Laravel Vapor apps, since AWS will kill any long-running processes (after 15 minutes). See here for an example.

Imagine you have a huge list of invoices in your DB, which need to be submitted one by one:

class SendInvoices implements ShouldQueue
{
    use InteractsWithQueue, Queueable;

    public function handle()
    {
        Invoice::pending()->cursor()
            ->takeUntilTimeout(Carbon::createFromTimestamp(LARAVEL_START)->add(14, 'minutes'))
            ->each(fn ($invoice) => $invoice->submit());

        if (Invoice::pending()->exists()) {
            $this->release();
        }
    }
}

This will submit as many invoices as there is time for, and then dispatch the job again to keep going.


If you can submit invoices in bulk, you can even chunk them before the takeUntilTimeout call:

Invoice::pending()->cursor()
    ->chunk(100)
    ->takeUntilTimeout(Carbon::createFromTimestamp(LARAVEL_START)->add(14, 'minutes'))
    ->each(fn ($invoices) => submit_invoices($invoices->all()));

P.S. This would've been ssoooo much easier to test with Carbon, especially with the new time travel stuff. But it makes no sense to add Carbon as a dependency to illuminate/collections just for this.

I really hate all that stupid Mockery code I had to write, but I see no other way. Gives you a new level of appreciation for proper time travel 😄

@clarkeash
Copy link
Contributor

@JosephSilber does it make sense to use DateTimeInterface instead so people can also use CarbonImmutable/ DateTimeImmutable as well?

@taylorotwell
Copy link
Member

Yeah probably want DateTimeInterface.

@JosephSilber
Copy link
Contributor Author

Done ✅

@taylorotwell taylorotwell merged commit 688da04 into laravel:8.x Sep 22, 2020
@JosephSilber JosephSilber deleted the take-until-timeout branch September 22, 2020 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants