Slow save times with Solspace Calendar and lots of events

[caseyreid]

Describe the bug

We're having an issue with saving and editing Calendar entries since we recently upgraded to Craft 3.7 and the latest version of Calendar and SEOMatic. We upgraded from Craft 3.4.

The issue is that it takes an extremely long time for a Calendar event to save, either when being created for the first time or when being edited in the control panel. We're talking at least 1-2 minutes after hitting the save button. Oftentimes this causes timeouts and out of memory errors for content editors.

This occurs in a single calendar called "Events" which has 2000+ event entries. It does not occur in a newly created calendar without any events, when a new event is created or saved.

In order to isolate the issue, I disabled various other plugins on the site and then performed the event save operation. When SEOMatic is disabled, the save operation performs quickly, and as expected. When it is enabled, that is when the slow save times happen.

That lead me to think that perhaps SEOMatic operations are taking an abnormally long time when event entry types are being saved, when lots of events are in the system.

To reproduce

Steps to reproduce the behaviour:

1. Save a new or existing Solspace Calendar Event entry with SEOMatic enabled

Expected behaviour

Saving an event entry should save in a reasonable amount of time when SEOMatic is enabled.

Screenshots

Versions

• Craft 3.7.13
• SEOMatic 3.4.14
• Solspace Calendar 3.3.5
• PHP 7.4.3
• MySQL 5.7 or 8

[khalwat]

As a test, if you go to SEOmatic -> Plugin Settings -> Advanced and turn OFF the Regenerate Sitemaps Automatically does the slowness continue?

[caseyreid]

That was it. Turning off that setting resolved the slowness.

In looking at the docs, it sounds like we'd either want to makes sure runQueueAutomatically is set to false and we have another queue processor in place, or we'd want to manually regenerate sitemaps using the console command, perhaps via cron?

[khalwat]

Well, I think there was a regression here somehow... either with SEOmatic or with Craft.

It should not be super-slow just regenerating a sitemap, it should run as a background queue process. Does this happen to you only in local dev, or in what environments does it happen?

How are your queue jobs being run? More context here: https://nystudio107.com/blog/robust-queue-job-handling-in-craft-cms

[khalwat]

It almost sounds to me like the entire request is waiting for it to finish regenerating the sitemap for the Events channel, which is not what should be happening.

[caseyreid]

This happens both locally (Nitro) and on our production server (ArcusTech). In both cases, we have runQueueAutomatically enabled. For the time being, I'm going to disable that setting on production in favor of manually running the queue via cron.

I too wonder if some regression has happened. We did not have this issue prior to upgrading from Craft 3.4 to 3.7 with basically the same number of events in the system.

[khalwat]

How are your queue jobs being run?

[caseyreid]

Just using Craft's default of someone accessing the CP.

[khalwat]

Okay so they are run via web request, that explains it a bit.

Addressed in: 9b81709

You can try it now by setting your semver in your composer.json to look like this:

    "nystudio107/craft-seomatic": "dev-develop as 3.4.14”,

Then do a composer clear-cache && composer update

[caseyreid]

I've updated locally to the dev-develop version. With Regenerate Sitemaps Automatically enabled, the slowness exists. With that setting disabled, save times are as expected.

So it doesn't look like that change necessarily did anything when sitemap regeneration is enabled.

[khalwat]

That's very odd, because the queue job should not be run automatically anymore.

[caseyreid]

Here is the results of various config settings when testing locally (Nitro).

Regenerate Sitemaps Automatically: enabled
runQueueAutomatically: true
Results of event entry save: slow
"Generating events sitemap" queue job generated: no

Regenerate Sitemaps Automatically: enabled
runQueueAutomatically: false
Results of event entry save: expected
"Generating events sitemap" queue job generated: yes

Regenerate Sitemaps Automatically: disabled
runQueueAutomatically: true
Results of event entry save: expected
"Generating events sitemap" queue job generated: yes

Regenerate Sitemaps Automatically: disabled
runQueueAutomatically: false
Results of event entry save: expected
"Generating events sitemap" queue job generated: yes

[khalwat]

What's happening, I think, is Craft is trying to run the queue job on the current request, so the Save doesn't finish until the sitemap queue job finishes rendering the queue (which can be quite lengthy)

...which is not how it used to work.

[khalwat]

@caseyreid can you confirm for me that src/helpers/Queue.php looks like this on your local install?

https://github.com/nystudio107/craft-seomatic/blob/develop/src/helpers/Queue.php#L56

Reason I ask is that I had someone else who was having a problem similar to yours where the above change fixed it for them.

[khalwat]

I'm fairly confident that this has been resolved in 3.4.14 -> https://github.com/nystudio107/craft-seomatic/releases/tag/3.4.14

If not, please let me know @caseyreid and we can screen share to diagnose.