[FR] Opt out of Project Config #4459

khalwat · 2019-06-28T05:41:04Z

If we're not using the project.yaml it'd be nice if we could opt out of Project Config.

The reason is that numerous pieces of data that used to exist in relational database tables is now in a big serialized blob in the info table.

If we start to get a large number of fields, the process of adding a new field is orders of magnitude slower than on Craft 3.0.x. And I have seen it result in making Craft unusable for adding new fields, resulting in PHP errors.

Additionally, just the logging of this massive blob as it is saved (with devMode on) causes severe slowdowns and PHP memory errors.

The text was updated successfully, but these errors were encountered:

engram-design · 2019-06-28T05:45:52Z

FWIW, I've noticed a pretty noticeable slowdown in editing fields, and sounds like this might be the issue.

I'm not sure how practical this is, as most things seem to be routed through project config, but I'd certainly be keen for this.

brandonkelly · 2019-06-28T13:48:51Z

It would be next to impossible to make the whole thing optional; anything that currently supports project config would need to start accounting for both scenarios.

That said there is likely room for improvement in how the config gets stored, so it’s more scalable to larger sites. Do either of you have a particularly painful example you’d be able to share with us? If so send the composer.json and composer.lock files, plus a database backup, over to [email protected].

Mosnar · 2019-06-28T14:06:45Z

Is there any particular reason the project config is serialized vs json_encoded? It doesn't seem like any full classes are being serialized, only array/object data. It would seem that PHP serialization typically takes additional computational effort and storage space. I naively switched the encoding to JSON and immediately saw a massive decrease in storage size as well as measurable speed improvements.

BenParizek · 2019-06-28T15:51:34Z

FWIW, I find myself generally frustrated with not being able to see simple things in the database any longer too. I often enable Project Config locally to tweak things on projects where we choose not to use it for the overall workflow. But it mostly just makes me feel like I need to take extra steps to accomplish simple things or get simple information while troubleshooting or testing in development.

andris-sevcenko · 2019-06-28T17:09:25Z

Is there any particular reason the project config is serialized vs json_encoded? It doesn't seem like any full classes are being serialized, only array/object data. It would seem that PHP serialization typically takes additional computational effort and storage space.

The reason is that a JSON value in the database would be tempting to some people to "just modify this one little bit, because it's faster to do by hand". Modifying that by hand could lead to a support ticket where we have no clear way of figuring out what went wrong.

A serialized value is a pretty good deterrent for that.

khalwat · 2019-06-28T18:54:45Z

That said there is likely room for improvement in how the config gets stored, so it’s more scalable to larger sites. Do either of you have a particularly painful example you’d be able to share with us? If so send the composer.json and composer.lock files, plus a database backup, over to [email protected].

@brandonkelly I submitted some logs to P&T via [email protected] last night where adding a field to a Matrix block causes it to either:

Take > 30s and succeed
Time out due to gateway timeouts from php-fpm and fail
Error when logging due to php-fpm trying to allocate something absolutely massive when attempting to log what's being saved to the info table

I can discuss with the client about getting you a db dump but it should be okay.

narration-sd · 2019-06-28T19:34:05Z

Andris, what you said here really made my ears prick up – an expression I hope goes with your over-the-top excellent English… I must reflect that I’ve stayed strictly away from Project Config, at least until it doesn’t generate a constant stream of issues, and yet its presence has already caused a few ‘blind’ hiccups when it didn’t like something I did during development. But it seems mainly that this is a tool strictly for large-public-view, heavily managed sites, and is more likely to be just a bother for less intense ones, as Ben for one indicates. The performance issue surfacing is one argument surely in that direction, and too bad if Brandon’s present take on this seems to mitigate decoupling/optioning it for lighter-weight Craft uses. But when I saw this about avoiding efficiency in order to trap rogue editors of internals, well… If you really care this way, and I can actually appreciate the dynamics, if it reads a bit like counter-tricks in an espionage novel, why not just light-weight encrypt the JSON others suggest, which would give you a blob sufficiently impenetrable for purpose, would seem? Doesn’t feel it would take many cpu cycles of such mangling to discourage edits other than in the Yaml where you want it? Best, Clive From: Andris Sevcenko Sent: Friday, June 28, 2019 10:09 AM To: craftcms/cms Cc: Subscribed Subject: Re: [craftcms/cms] [FR] Opt out of Project Config (#4459) Is there any particular reason the project config is serialized vs json_encoded? It doesn't seem like any full classes are being serialized, only array/object data. It would seem that PHP serialization typically takes additional computational effort and storage space. The reason is that a JSON value in the database would be tempting to some people to "just modify this one little bit, because it's faster to do by hand". Modifying that by hand could lead to a support ticket where we have no clear way of figuring out what went wrong. A serialized value is a pretty good deterrent for that. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

brandonkelly · 2019-06-28T21:46:47Z

We just profiled saving a field before and after switching to JSON-encoding the config, and saw a 5.2x performance gain, so yeah, that settles that.

Swapped to JSON for Craft 3.1.33 and 3.2.0-RC3.

To get the fix early, change your craftcms/cms requirement in composer.json to one of these, depending on whether you’re running Craft 3.1 or the 3.2 RC:

"require": {
  "craftcms/cms": "dev-develop#6525f43f86357d32ebd0bb4838217a9ec223a24e as 3.1.32",
  "craftcms/cms": "3.2.x-dev#3449e30688d06d64e85017d9db7861c3635a029f as 3.2.0-RC2",
  "...": "..."
}

Then run composer update.

There’s still likely some room for improvement, so going to leave this open. For 3.3 we are going to look into storing the project config in its own config table with path and value columns so we only need to update the things that changed, and keep the full config stored in cache.

narration-sd · 2019-06-28T22:14:24Z

Just totally cool, @brandonkelly!

…

On Fri, Jun 28, 2019, 14:46 Brandon Kelly ***@***.***> wrote: We just profiled saving a field before and after switching to JSON-encoding the config, and saw a 5.2x performance gain, so yeah, that settles that. Swapped to JSON for Craft 3.1.33 and 3.2.0-RC3. To get the fix early, change your craftcms/cms requirement in composer.json to one of these, depending on whether you’re running Craft 3.1 or the 3.2 RC: "require": { "craftcms/cms": "dev-develop#6525f43f86357d32ebd0bb4838217a9ec223a24e as 3.1.32", "craftcms/cms": "3.2.x-dev#3449e30688d06d64e85017d9db7861c3635a029f as 3.2.0-RC2", "...": "..." } Then run composer update. There’s still likely some room for improvement, so going to leave this open. For 3.3 we are going to look into storing the project config in its own config table with path and value columns so we only need to update the things that changed, and keep the full config stored in cache. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4459?email_source=notifications&email_token=AAB4RCPU3KG72S7HJA3TZ7LP42BERA5CNFSM4H4CJML2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY3HURQ#issuecomment-506886726>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAB4RCKIPAVIOBVUFOP3GDTP42BERANCNFSM4H4CJMLQ> .

khalwat · 2019-06-29T02:33:13Z

So this is definitely an improvement @brandonkelly -- in local dev, I'm able to save a new Text field in a Matrix block in 02:05:38 -- which is significantly faster than before. But still, 2 minutes to add a new field (albeit in local dev) seems like a lot.

It's also generating 504.8 MB in log files spanning 6 web.log files for this single request (I deleted the logs prior to clicking on Save to add a new field in a Matrix block), and I still end up with the php error:

[28-Jun-2019 19:21:41 America/Los_Angeles] PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 6864896 bytes) in /var/www/app/vendor/yiisoft/yii2-debug/src/LogTarget.php on line 60

I believe the error, and at least part of the slowdown is from logging. Here's a partial fragment of the query log:

2019-06-28 19:20:50 [-][4017][-][info][yii\db\Command::execute] UPDATE `craft_info` SET `id`=1, `version`='3.1.32.1', `schemaVersion`='3.1.28', `maintenance`=0, `config`='{\"dateModified\":1561774368,\"siteGroups\":{\"735230a9-5d55-4d8f-9291-350fa08cf1c6\":{\"name\":\"XXX (en-US)\"}},\"sites\":{\"bbed078a-e3fa-4b6f-b6ed-c228c35f55a9\":{\"baseUrl\":\"https://XXX.com\",\"handle\":\"en_us\",\"hasUrls\":\"1\",\"language\":\"en-US\",\"name\":\"XXX (en-US)\",\"primary\":\"1\",\"siteGroup\":\"735230a9-5d55-4d8f-9291-350fa08cf1c6\",\"sortOrder\":\"1\"}},\"sections\":{\"005913ec-e85a-4563-99a2-99a7cb278409\":{\"enableVersioning\":\"1\",\"entryTypes\":{\"db92bde7-da6e-41a0-af10-47142cc18f1d\":{\"fieldLayouts\":{\"d293bcab-f2fd-49bf-88f1-9ecf06e8d0c8\":{\"tabs\":[{\"fields\":{\"84bedb51-2a25-448f-9f0f-89d9828babc5\":{\"required\":\"0\",\"sortOrder\":\"2\"},\"a785ae32-cb19-4b83-ab15-4b4b4bb59183\":{\"required\":\"0\",\"sortOrder\":\"1\"}},\"name\":\"Default\",\"sortOrder\":

Due to the number of fields, it's trying to log a gargantuan amount of data here, and finally gives up.

khalwat · 2019-06-29T03:08:45Z

If it's helpful... this site has 252 fields, many of which are Matrix fields that contain sub-fields, often SuperTable fields... which then have their own fields.

So there's probably more config data than normal for just 252 straight up fields.

angrybrad · 2019-06-29T03:38:49Z

@khalwat i'm assuming that's with devMode enabled, correct? Out of curiosity, what sort of numbers do you see with it disabled?

khalwat · 2019-06-29T14:24:56Z

Correct @angrybrad -- I will try it without devMode on for you.

khalwat · 2019-06-29T14:35:59Z

With devMode off, it completes adding a new field in 00:40:65 which is definitely better... but we still get a phperrors.log with:

[29-Jun-2019 07:32:20 America/Los_Angeles] PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 99442688 bytes) in /var/www/app/vendor/yiisoft/yii2-debug/src/LogTarget.php on line 60

And in the web.log I'm seeing a bunch of:

2019-06-29 07:31:49 [-][4017][-][warning][craft\db\mysql\Schema::releaseSavepoint] Tried to release a savepoint, but it does not exist: SQLSTATE[42000]: Syntax error or access violation: 1305 SAVEPOINT LEVEL4 does not exist
The SQL being executed was: RELEASE SAVEPOINT LEVEL4
2019-06-29 07:31:50 [-][4017][-][warning][craft\db\mysql\Schema::releaseSavepoint] Tried to release a savepoint, but it does not exist: SQLSTATE[42000]: Syntax error or access violation: 1305 SAVEPOINT LEVEL3 does not exist
The SQL being executed was: RELEASE SAVEPOINT LEVEL3
2019-06-29 07:31:50 [-][4017][-][warning][craft\db\mysql\Schema::releaseSavepoint] Tried to release a savepoint, but it does not exist: SQLSTATE[42000]: Syntax error or access violation: 1305 SAVEPOINT LEVEL2 does not exist
The SQL being executed was: RELEASE SAVEPOINT LEVEL2
2019-06-29 07:31:50 [-][4017][-][warning][craft\db\mysql\Schema::releaseSavepoint] Tried to release a savepoint, but it does not exist: SQLSTATE[42000]: Syntax error or access violation: 1305 SAVEPOINT LEVEL2 does not exist
The SQL being executed was: RELEASE SAVEPOINT LEVEL2

...which is odd, and makes me wonder if something else is afoot here as well.

It does complete adding the field successfully, however.

Mosnar · 2019-06-29T15:11:23Z

@khalwat Could you share the byte size of your config column out of curiosity?

khalwat · 2019-06-29T19:38:33Z

@Mosnar So it looks like it's actually not that large in the db:

mysql> SELECT sum(char_length(config))/1024/1024 FROM craft_info;
+------------------------------------+
| sum(char_length(config))/1024/1024 |
+------------------------------------+
|                         1.11367702 |
+------------------------------------+

about 1.1 MB so there must be something else going on in these logs, I will have a deeper look. The really large tables are:

+--------------------------------------------------------------+-----------+
| Table                                                        | Size (MB) |
+--------------------------------------------------------------+-----------+
| craft_entryversions                                          |       162 |
| craft_audit_log                                              |       128 |
| craft_retour_stats                                           |        63 |
| craft_searchindex                                            |        18 |
| craft_content                                                |        15 |
| craft_elements                                               |         6 |

nettum · 2019-07-01T07:13:56Z

Not sure if it's related, but I'm experiencing the same issue as @khalwat with slow save-times when adding new fields to a matrix (30sec+).
I did some digging earlier and for me the most timeconsuming thing seems to be alot of calls to containsMb4 with the data from info.config table when saving a matrix.
For me craft did 142 update queries to the info table which result in 142 calls to containsMb4 on save matrix field - so alot of data going through the wire here. As far as I can see the calls to containsMb4 only happens when running mysql, not postgres.
P&T, if interested - more detailed information in email to [email protected], initiated 24.05.2019 13:50

brandonkelly · 2019-07-01T15:13:06Z

@nettum Thanks for pointing that out! Just fixed that for the next release. Guessing that will drastically improve your site’s performance as well @khalwat.

To get the fix early, change your craftcms/cms requirement in composer.json to one of these, depending on whether you’re running Craft 3.1 or the 3.2 RC:

"require": {
  "craftcms/cms": "dev-develop#048388f4243e5451e0b2c203cd556c04130fdb8f as 3.1.32",
  "craftcms/cms": "3.2.x-dev#64253d90ae56ac7a5d328e1a59c445cfee27d69a as 3.2.0-RC2",
  "...": "..."
}

Then run composer update.

khalwat · 2019-07-01T23:58:56Z

With devMode off and the latest build, it completes adding a new field in 00:29:18 which is getting down to the realm of reasonable-ness. And given that it's in local dev inside of Docker, I think on a live production server it'll be quite usable.

Also there is no phperrors.log created for this build.

Nice work @brandonkelly ! Will look forward to Project Config not being a big blob in the future, but this is a massive improvement!

StudioShand · 2019-07-03T14:05:21Z

I'm glad I came across this thread. I'm in the middle of porting a site over from Expression Engine and it's starting to drive me crazy the time it's taking adding new fields in Craft! It's definitely taking longer and longer as I add more fields and like nothing I've ever experienced in EE. This is with devMode off on a production server. I'm also noticing a significant lag when loading related entries, I doubt that is related though. Hopefully the most recent update will improve things.

khalwat · 2019-07-03T14:42:31Z

For anyone reading this, the fix was rolled into Craft CMS 3.1.33 -> https://github.com/craftcms/cms/blob/develop/CHANGELOG-v3.md#3133---2019-07-02

gregmartyn · 2019-11-19T23:03:42Z

6525f43 was a forward incompatible change with a minor version bump. If you rollback to 3.1.32.1 after running 3.1.33, you'll get yii\base\ErrorException: unserialize(): Error at offset 0 of 141549 bytes in /var/www/app/craft/vendor/craftcms/cms/src/services/ProjectConfig.php:1119

andris-sevcenko · 2019-11-20T08:04:49Z

@gregmartyn you should also be restoring the database if rolling back, since migrations (if any), cannot be reverted and, if there have been any DB schema changes, Craft, in general, will refuse to run.

gregmartyn · 2019-11-20T14:22:44Z

Sorry, I'm still new to Craft. This is surprising behavior to me. I'd expect at least the minor x.Y.z version to change when the database is incompatible. Migrations have degrees of compatibility, but a change like this where a table is rewritten in a different format that the previous patch-level version is incompatible with -- and affects all requests -- is pretty major.

brandonkelly · 2019-11-20T14:39:10Z

@gregmartyn We’ve never thought about it that way, but we’re in the process of working out a more formal release schedule & guidelines internally, and this is great feedback. I like the idea of ensuring that all patch release migrations (x.y.Z) should be revertable, and if that’s not possible, it should be held off until the next minor release (x.Y.0).

And sorry about the inconvenience here. Is there a reason you are trying to revert in the first place? The migration you are referring to was a pretty significant performance improvement.

gregmartyn · 2019-11-20T16:31:01Z

We launched some code to production that had a minor bug in it. (It affected only the admin area) The launch also included upgrading the patch level of Craft. (x.y.Z)

When we rolled back to the previously deployed Docker image of the site, the entire site went down. We were aware of the Craft version number change, but it wasn't immediately obvious that the two versions were incompatible. The version number change only suggested that the newer version might have some bugfixes, so it looked like our database data had been corrupted. (our bug was being triggered by other unrelated data missing) Even looking back at the Craft changelog now it's not obvious that there were changes to the database. A call out saying that there were db changes may have helped, but really it was the incompatibility of two patch x.y.Z releases that threw us off.

Thanks for listening!

brandonkelly added project config 🐫 features related to Project config enhancement improvements to existing features labels Jun 28, 2019

brandonkelly added a commit that referenced this issue Jun 28, 2019

JSON-encode the project config (#4459)

6525f43

brandonkelly added a commit that referenced this issue Jul 1, 2019

Improved field saving performance (#4459)

048388f

khalwat closed this as completed Jul 2, 2019

engram-design mentioned this issue Jul 14, 2019

Saving Matrix - SuperTable fields results in a save time period as-long-as Apache2 timeout value verbb/super-table#71

Closed

wihodges mentioned this issue Aug 8, 2019

supertable field not saving, causing memory/backend crash. verbb/super-table#292

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FR] Opt out of Project Config #4459

[FR] Opt out of Project Config #4459

khalwat commented Jun 28, 2019

engram-design commented Jun 28, 2019

brandonkelly commented Jun 28, 2019 •

edited

Loading

Mosnar commented Jun 28, 2019 •

edited

Loading

BenParizek commented Jun 28, 2019

andris-sevcenko commented Jun 28, 2019

khalwat commented Jun 28, 2019

narration-sd commented Jun 28, 2019 via email

brandonkelly commented Jun 28, 2019

narration-sd commented Jun 28, 2019 via email

khalwat commented Jun 29, 2019 •

edited

Loading

khalwat commented Jun 29, 2019 •

edited

Loading

angrybrad commented Jun 29, 2019

khalwat commented Jun 29, 2019

khalwat commented Jun 29, 2019

Mosnar commented Jun 29, 2019

khalwat commented Jun 29, 2019

nettum commented Jul 1, 2019

brandonkelly commented Jul 1, 2019

khalwat commented Jul 1, 2019 •

edited

Loading

StudioShand commented Jul 3, 2019

khalwat commented Jul 3, 2019

gregmartyn commented Nov 19, 2019

andris-sevcenko commented Nov 20, 2019

gregmartyn commented Nov 20, 2019

brandonkelly commented Nov 20, 2019

gregmartyn commented Nov 20, 2019

[FR] Opt out of Project Config #4459

[FR] Opt out of Project Config #4459

Comments

khalwat commented Jun 28, 2019

engram-design commented Jun 28, 2019

brandonkelly commented Jun 28, 2019 • edited Loading

Mosnar commented Jun 28, 2019 • edited Loading

BenParizek commented Jun 28, 2019

andris-sevcenko commented Jun 28, 2019

khalwat commented Jun 28, 2019

narration-sd commented Jun 28, 2019 via email

brandonkelly commented Jun 28, 2019

narration-sd commented Jun 28, 2019 via email

khalwat commented Jun 29, 2019 • edited Loading

khalwat commented Jun 29, 2019 • edited Loading

angrybrad commented Jun 29, 2019

khalwat commented Jun 29, 2019

khalwat commented Jun 29, 2019

Mosnar commented Jun 29, 2019

khalwat commented Jun 29, 2019

nettum commented Jul 1, 2019

brandonkelly commented Jul 1, 2019

khalwat commented Jul 1, 2019 • edited Loading

StudioShand commented Jul 3, 2019

khalwat commented Jul 3, 2019

gregmartyn commented Nov 19, 2019

andris-sevcenko commented Nov 20, 2019

gregmartyn commented Nov 20, 2019

brandonkelly commented Nov 20, 2019

gregmartyn commented Nov 20, 2019

brandonkelly commented Jun 28, 2019 •

edited

Loading

Mosnar commented Jun 28, 2019 •

edited

Loading

khalwat commented Jun 29, 2019 •

edited

Loading

khalwat commented Jun 29, 2019 •

edited

Loading

khalwat commented Jul 1, 2019 •

edited

Loading