Blocks lost when provisional draft is applied

[ccchapman]

Bug Description

We are experiencing content loss on the latest version of all packages. The problem is intermittent and affects only some entries. We discovered the Craft CLI resave action is a temporary band-aid however the issue will come back.

ℹ️ Note: The issue can be reproduced with Typed Link Field enabled or disabled. Unsure if this makes Typed Link Field is irrelevant.

Potentially related issues:

• Issue when update block #718
• Compatiblity issue with Neo, all blocks get randomly deleted if one or more contain a typed link field sebastian-lenz/craft-linkfield#233

I will email a video, DB, and composer.lock.

Thanks! 😄

Steps to reproduce

1. Edit the entry admin/entries/reusableContent/403150
2. Scroll to bottom of entry
3. Change some text
4. Wait for provisional draft to save
5. CMD + S

Expected behaviour

Save w/o blocks going missing

Neo version

3.7.0

Craft CMS version

4.4.1

What is the affected Neo field's propagation method?

Field propagation method is all, section propagation method is siteGroup

Does this issue involve templating, and if so, is eager-loading used?

This is not a templating issue

[mfell]

Thanks for your bug report ... I had a similar effect, after updating Neo to 3.7.0.

It accoured under Craft CMS 4.3.10 (or 11) ... unfortunaly I had no idea what to do and because of this I keep staying on Neo 3.6.4

If needed: I can email a video, DB, and composer.lock, too.

[ttempleton]

Thanks for sending that email @ccchapman. Could you please also send in the composer.json and I'll have a look. Unfortunately I'm unable to reproduce this on my usual local testing environment... and also pretty curious about it seemingly only affecting 3.7.0. I can't think of any change in 3.7.0 that would cause this, but could definitely be wrong about that.

[ccchapman]

@ttempleton,

Thanks for your reply.

• Will send composer.json now!
• My issue was not limited to 3.7.x. It occurred in Neo 3.5.x too. I had upgraded to see if it would be fixed in a newer version.
• Take with a grain of salt, but I think the issue started happening much more frequently after we added a second site to the system.

[ccchapman]

craftcms/cms#12858 looks possibly related? I can try to test that tomorrow.

[ccchapman]

craftcms/cms@031bfe8 does not address this issue.

[mfell]

Hi ... I just checked this issue with Craft 4.4.1 and Neo 3.7.1 -> blocks are lost.

I'm unsure, how to support your work, but I can send you my composer.json, DB, Server information ... whatever you need.

With Neo 3.6.4 -> everything's fine ... in my case.

[ttempleton]

Thanks for sending that in @ccchapman. It looks like all of the blocks that disappear have their primaryOwnerId set to a provisional draft, which is pretty strange since the primaryOwnerId-setting should work the same as with Matrix blocks, and I haven't heard of any such bug affecting Matrix blocks. I'm going to try different actions to trigger the initial incorrect setting, and to figure out how best to set already affected blocks to the correct primaryOwnerId. Will update here when I have more info.

@mfell if it's only occurring for you with 3.7.0+ then I'm wondering if you have a different issue. Could you please send your composer.json/lock files and composer backup to [email protected] and we'll have a look at it.

[mfell]

Hi again, I'll send @ttempleton the needed files in a few minutes, a few words here:

Craft 4.4.2, Neo 3.7.1 (Update from 3.6.4)

The child-entries are lost (existing entries)

The relation parent / child of new entries is lost -> correct: has to be nested

Some server config (ddev)

[sfsmfc]

We have a similar error. After provisional draft is finished, we have wrong child elements. In some cases, the "new wrong" element has already content. If we store the whole entry, we can see the right elements.

This happens, if we use Neo in Version 3.7.x. If we downgrade to version 3.6.4, all is fine.

@ttempleton If you need more informations, please let me know.

[vebjorn]

We have a similar error. After provisional draft is finished, we have wrong child elements. In some cases, the "new wrong" element has already content. If we store the whole entry, we can see the right elements.

I have the exact same issue. To get the right content we don't have to save the entry though. Reloading the page, which then will pull in the draft fresh, will give the correct data. Almost seems like a JS-refresh-bug or something

[ttempleton]

Thanks for sending those files @mfell. I was able to reproduce the second part of your report where the automatically-created child blocks were being created at the wrong level, and that should be fixed now in Neo 3.7.2. I was unable to reproduce the block loss from your first screenshot - is there any action in particular that causes it for you?

@vebjorn @sfsmfc that sounds like you are being affected by #721.

[mfell]

@ttempleton great, thank you so much!

I think this might be the same cause ... let me check this with Neo 3.7.2, I'll send you feedback soon as possible. Today I'm out of office, but during the weekend I'll do it.

[mfell]

Ok, hi @ all :)

Just checked this with Craft 4.4.3, Neo 3.7.3 under MacOS 12.6.x and Chrome:

• the bad news: this bug is still accouring
• the good news: no content is lost

After creating a new element (1):

• the new parent / child relationship is now correct
• but existing elements lost their children

After page reload (2):

• the parents have their lost children again (yeah)
• but it's not possible to save the page, of course

(1)

(2)

[mfell]

Hi again,

just popped up in my mind: I didn't cleaned the browser cache yesterday.

I just checked this issue again with Craft 4.4.3, Neo 3.7.3 under MacOS 12.6.x, Chrome AND deleted browser cache:
It seems to be ok now ... at least in my case.

Please: May anyone confirm this?

[ccchapman]

I confirmed the issue described in the original comment persists on Neo 3.7.3.

Composer info

craftcms/aws-s3                            2.0.3            Amazon S3 integration for Craft CMS
craftcms/cms                               4.4.3            Craft CMS
craftcms/feed-me                           5.1.1            Import content from XML, RSS, CSV or JSON feeds into entries, categories, Craft Commerce products, and more.
craftcms/redactor                          3.0.4            Edit rich text content in Craft CMS using Redactor by Imperavi.
diginov/craft-sentry-logger                4.1.1            Pushes Craft CMS logs to Sentry through a real Yii 2 log target
hybridinteractive/craft-position-fieldtype v4.x-dev f9c0ef9 Brings back the Position fieldtype from Craft 2
korcontrol/craft-classy                    2.0.0            Twig helpers inspired by https://github.com/JedWatson/classnames
mutation/translate                         3.0.1            Translate messages in the control panel
nystudio107/craft-closure                  1.0.0            Allows you to use arrow function closures in Twig
nystudio107/craft-retour                   4.1.11           Retour allows you to intelligently redirect legacy URLs, so that you don't lose SEO value when rebuilding & restructuring ...
nystudio107/craft-seomatic                 4.0.22           SEOmatic facilitates modern SEO best practices & implementation for Craft CMS 4. It is a turnkey SEO system that is compre...
nystudio107/craft-vite                     4.0.5            Allows the use of the Vite.js next generation frontend tooling with Craft CMS
putyourlightson/craft-blitz                4.4.3            Intelligent static page caching for creating lightning-fast sites.
sebastianlenz/linkfield                    2.1.5            A Craft field type for selecting links
spacecatninja/imager-x                     4.1.11           Ninja powered image transforms.
spicyweb/craft-embedded-assets             3.1.1            Manage YouTube videos, Instagram photos, Twitter posts and more as first class assets
spicyweb/craft-neo                         3.7.3            A Matrix-like field type that uses existing fields
topshelfcraft/environment-label            4.0.2            ...so you don't forget where you are.
topshelfcraft/wordsmith                    4.2.0            ...because you have the best words.
verbb/navigation                           2.0.16           Create navigation menus for your site.
vlucas/phpdotenv                           v5.5.0           Loads environment variables from `.env` to `getenv()`, `$_ENV` and `$_SERVER` automagically.
yiisoft/yii2-redis                         2.0.18           Redis Cache, Session and ActiveRecord for the Yii framework
yiisoft/yii2-shell                         2.0.5            The interactive shell extension for Yii framework

[ttempleton]

@mfell when that was happening, was there anything in particular you were editing that could have led to that error, or did it just always happen no matter what you changed on the entry? I was never able to reproduce that part of your report, so maybe it was just some sort of caching issue if clearing the cache has fixed it for you.

[mfell]

@ttempleton It happens always on save ... I didn't notice any special action witch makes this effect accouring.

And in my case, it's working well now ... I can't report any problem for now. (Thanks a lot!)

[ccchapman]

This issue has now occurred on another client. Their frontend pages were missing content. When those pages were then edited in Craft following the steps here, blocks would disappear.

This project (also multi-site) is experiencing the same behaviours as reported in original comment.

We confirmed that the affected Neo blocks had a primaryOwnerId of a provisional draft. craft entries/resave will fix the primaryOwnerId to be the currently published entry ID.

We are still unclear what causes the primaryOwnerId to be set incorrectly in the first place.

We have tested with a "good" version of the DB to check if GC triggers the issue; however, it does not appear relevant.

The provisional draft which was set as the primaryOwnerId exists in the "good" DB too. Therefore I do not believe the creation of the provisional is triggering the bad assignment?

This project has different package versions:

• Craft Pro 4.2.7
• PostgreSQL 14.3
• Neo 3.4.1

I can provide more (incl. "good" vs. "bad" DB) as required.

[ttempleton]

@ccchapman Does that project use any Matrix fields? If so, do any Matrix blocks have an incorrect primaryOwnerId? I haven't been able to reproduce the incorrect primaryOwnerId on projects other than what had already occurred on the database backup you sent in, and I can't find any difference in Neo vs. Matrix field code that would cause Neo and not Matrix to have the incorrect primaryOwnerId set. Note that Matrix fields would be unlikely to suffer block loss in this case, as it seems to be the Neo block structure saving in particular that loses the Neo blocks, due to the structure saving not being given the data for the blocks with the incorrect primaryOwnerId.

[mfell]

Hi, I have no idea whether this comment is helpful or not ... but this might be a JS topic and during my redactorial editing with Neo under my local dev environment, I got this (screenshot).

If this post is not useful, please feel free to ignore or delete it. :)

[ccchapman]

@ccchapman Does that project use any Matrix fields? If so, do any Matrix blocks have an incorrect primaryOwnerId? I haven't been able to reproduce the incorrect primaryOwnerId on projects other than what had already occurred on the database backup you sent in, and I can't find any difference in Neo vs. Matrix field code that would cause Neo and not Matrix to have the incorrect primaryOwnerId set. Note that Matrix fields would be unlikely to suffer block loss in this case, as it seems to be the Neo block structure saving in particular that loses the Neo blocks, due to the structure saving not being given the data for the blocks with the incorrect primaryOwnerId.

Yes, the project does have entries with Matrix fields. I had a look and all of those Matrix Blocks have a primaryOwnerId of the current entry ID as expected.

A couple other facts I found today:

• There were ~500 Neo Blocks with a "date updated" in a one minute span of one the issue is suspected to have occurred
• The issue seems to have affected a large number of entries/Neo fields during this one minute span
• The Neo Blocks with the bad assignment were not new blocks rather blocks which existed for many months and were updated to have the wrong primaryOwnerId

[ttempleton]

I'm still unable to reproduce it. Are there any clues you can find as to what operations were being performed by the Craft project around that minute span, that could have led to the Neo blocks being saved with the wrong primaryOwnerId?

[ccchapman]

I'm still unable to reproduce it. Are there any clues you can find as to what operations were being performed by the Craft project around that minute span, that could have led to the Neo blocks being saved with the wrong primaryOwnerId?

I have reviewed all available logs and did not find anything suspicious around that minute span. In fact the only activity then is the Craft session-info requests.

This makes me wonder if some kind of queue job (e.g. resave, propagation, saving Neo block structures for duplicated elements, etc.) happened around that time and triggered the issue. Unfortunately we do not have the stdout for the queue workers currently writing to a log. I do not have a way to know exactly which jobs would have been running at this time.

[ttempleton]

This turned out to be the Neo equivalent to craftcms/cms#13155. I've applied the fix for craftcms/cms#13181 to Neo, and have released Neo 3.7.9 with that fix.