Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional psi configurations #695

Closed
jonathantredway opened this issue Sep 22, 2021 · 2 comments · Fixed by #696
Closed

Additional psi configurations #695

jonathantredway opened this issue Sep 22, 2021 · 2 comments · Fixed by #696
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@jonathantredway
Copy link
Contributor

Is your feature request related to a problem? Please describe.

  1. I'm attempting to run the psi cron job against all of our public pages on prod. We have a high number of them (~40), so sending all of the fetch requests in parallel overwhelms the pagespeed insights api and tanks out with the error PSI collection failure for Production LHCI Cron Job - Home: PSI Failed (500): Lighthouse returned error: Something went wrong. (I've been able to replicate this outside of this package). I'd like to be able to limit the number of parallel requests sent to the pagespeed insights api.
  2. Also, a few of our pages error out when requesting the pwa category. I'd like to be able to opt out of categories from the config.

The above could be 2 separate issues, but I've grouped them together as these could both be optional psi configurations. While we're here, we could make other details configurable (see #420)

Describe the solution you'd like
I think the config could look something like:

psiCollectCron: {
    ...currentPsiCollectCronConfig,
    sites: [
        {
            ...currentSiteConfig,
            maxNumberOfParallelUrls: number, // default: Infinity (run all urls in parallel - current behavior)
            excludeCategories: ['performance' | 'accessibility' | 'best-practices' | 'pwa' | 'seo'], // default: [] (run all categories - current behavior)
            strategy: 'desktop' | 'mobile' // default: 'mobile' (current behavior)
        },
        ...etc
    ]
}

It wouldn't be difficult to set up an async counter to ensure a max number of requests are being processed in parallel. Even easier to just pass config values to each run.

Describe alternatives you've considered
Without being able to configure the number of parallel calls (hardcoded here) or being able to configure the categories (hardcoded here), I don't see alternatives aside from setting up our own implementation of this. Not that this would be terribly difficult, but I'd rather make these few pieces configurable since it would be almost identical to this package.

Additional context
Happy to open a PR, just wanted to get a pulse on if there is any opposition and why.

@patrickhulce
Copy link
Collaborator

Thanks for filing @jonathantredway! Very open to making more of this configurable and would be happy to review a PR for it (with a test or two 😉 )

A few thoughts:

It wouldn't be difficult to set up an async counter to ensure a max number of requests are being processed in parallel.

Even easier then that :) We already have bluebird in our deps, just use Bluebird.map(async () => {}, {concurrency: options.maxNumberOfParallelUrls})

excludeCategories: ['performance' | 'accessibility' | 'best-practices' | 'pwa' | 'seo'], /

Let's flip this around to categories and default to ['performance', 'accessibility', 'best-practices', 'seo', 'pwa'] :)

@jonathantredway
Copy link
Contributor Author

jonathantredway commented Sep 24, 2021

Opened a PR here 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants