[Conversion] Write down a summary of changes

[Reinmar]

Why

#10294 came with a large number of changes that are hard to discover without going really deep into the issue tracker.

Currently, the only place where we could send people is #11211.

What

Let's write a blog-post-like (developer-oriented) summary of what has changed including:

• The "whys" behind each of the change
• The "whats"

The idea is to make it digestible so we should:

• Limit ourselves to what matters to people
• Compile multiple smaller thoughts into topics (avoid explaining every single small change independently)

[Reinmar]

• Option 1: For the time being, let's put this in the migration guide (to be done in [Conversion] Add information about breaking changes in the conversion system to the migration guide #11188). Once the blog post is published, we'll remove this content from the migration guide and link to the blog post.
• Option 2: Write the summary here and link to this post from the Migration guide. The blog post may then take inspiration from this summary (but generalize shorten it even more).

Decision: Option 2.

[Reinmar]

Some ideas:

• If possible, come up with examples that weren't possible before and now are much easier. Potentially, the "before" will need to be written in pseudo-code
• We could link (or use) to some old examples in https://ckeditor.com/docs/ckeditor5/latest/framework/guides/deep-dive/conversion/element-reconversion.html.

Some of the important new features:

• Support for triggering an e2E converter based on attributes of the element – replaces multiple smaller converters (e2E + a2A or the case with <hx> based on <heading level=x>) and triggerBy.
• The entire e2S. Previously required playing with doubled-binding between single model element and multiple view).

[dawidurbanski]

Draft of the document: (internal) https://www.notion.so/Summary-of-changes-in-the-reconversion-system-3ed675d4e18249b8a9ca9bce94dcb04c

[dawidurbanski]

Summary of changes

To see all changes in v33.0.0 check the migration guide

Downcast conversion helpers and API changes

New downcast conversion helper: elementToStructure()

We have found that in many cases you may want to create a single view element with one or many children elements (a structure) in one go.

For this purpose we introduced a new downcast conversion helper: elementToStructure().

Previously you would probably use the elementToElement() conversion helper to perform this task like such:

// Convert <horizontalLine> model element
// into <div> <hr /> </div> structure.
editor.conversion.for( 'dataDowncast' ).elementToElement( {
	model: 'horizontalLine',
	view: ( modelElement, { writer } ) => {
		const viewWrapperElement = writer.createContainerElement( 'div' );
		const viewHrElement = writer.createEmptyElement( 'hr' );

		writer.insert( writer.createPositionAt( viewWrapperElement, 0 ), viewHrElement );
				
		return viewWrapperElement;
	}
} );

Currently this is discouraged and whenever possible you should use elementToStructure() instead.

Additionally we added an option to pass an array of children elements to writer.createContainerElement() in order to further simplify the process:

// Convert <horizontalLine> model element
// into <div> <hr /> </div> structure.
editor.conversion.for( 'dataDowncast' ).elementToStructure( {
	model: 'horizontalLine',
	view: ( modelElement, { writer } ) => {
		return writer.createContainerElement( 'div', null, [
			writer.createEmptyElement( 'hr' )
		] );
	}
} );

New, seamless reconversion mechanism

The reconversion API has been in beta for a while now. With CKEditor 33.0.0, it finally reached the final state. There are some notable changes to it.

Removal of the triggerBy option

Starting from CKEditor 23.1.0 you were able to replace a set of atomic converters (elementToElement() + attributeToAttribute()) with one converter handling any element + attributes combo using triggerBy option (see more).

// Convert <myElement type="single" ownerId="1"> model element
// into <div class="my-element my-element-single" data-owner-id="1"></div>.
editor.conversion.for( 'downcast' ).elementToElement( {
	model: 'myElement',
	view: ( modelElement, { writer } ) => {
		return writer.createContainerElement( 'div', {
			 'data-owner-id': modelElement.getAttribute( 'ownerId' ),
			class: `my-element my-element-${ modelElement.getAttribute( 'type' ) }`
		} );
	},
	triggerBy: {
		attributes: [ 'ownerId', 'type' ]
	}
} );

In CKEditor 33.3.0 we changed this API and from now on you should use attributes property on a model instead.

// Convert <myElement type="single" ownerId="1"> model element
// into <div class="my-element my-element-single" data-owner-id="1"></div>.
editor.conversion.for( 'downcast' ).elementToElement( {
	model: {
		name: 'myElement',
		attributes: [ 'ownerId', 'type' ]
	},
	view: ( modelElement, { writer } ) => {
		return writer.createContainerElement( 'div', {
			'data-owner-id': modelElement.getAttribute( 'ownerId' ),
			class: `my-element my-element-${ modelElement.getAttribute( 'type' ) }`
		} );
	}
} );

In the above, the <myElement> model element will be reconverted every time either ownerId or type is updated.

Additionally you can react to changes in the element children using children property.

// Convert <myElement><paragraph>A</paragraph></myElement> model element
// into <div class="my-element" data-type="single">...</div>.
// But if more paragraph are added like so:
// <myElement><paragraph>A</paragraph><paragraph>B</paragraph></myElement>
// then convert into <div class="my-element" data-type="multiple">...</div>.
editor.conversion.for( 'downcast' ).elementToElement( {
	model: {
		name: 'myElement',
		children: true
	},
	view: ( modelElement, conversionApi ) => {
		const { writer } = conversionApi;

		return writer.createContainerElement( 'div', {
			class: 'my-element',
			'data-type': modelElement.childCount == 1 ? 'single' : 'multiple'
		} );
	}
} );

Both attributes and children can be used together and are available in both elementToElement() and elementToStructure() conversion helpers.

The brand new slots API

Until now the “slots” concept was only briefly discussed (see: #1589). With the addition of the elementToStructure() conversion helper it became apparent, that we need a proper slots API in place.

Slots will allow you to place some view elements in one container element while putting some other element in another one.

For example:

editor.conversion.for( 'dataDowncast' ).elementToStructure( {
	model: {
		name: 'items',
		attributes: [ 'highlightedItems' ]
	},
	view: ( modelElement, { writer } ) => {
		const highlightedItems = modelElement.getAttribute( 'highlightedItems' ) || 0;

		return writer.createContainerElement( 'div', { class: 'items' }, [
			writer.createContainerElement( 'div', { class: 'highlighted' }, [
				writer.createSlot( element => element.index < highlightedItems )
			] ),
			writer.createContainerElement( 'div', null, [
				writer.createSlot( element => element.index >= highlightedItems )
			] )
		] );
	}
} );

Will result in such conversion:

Model:

<items highlightedItems="2">
	<paragraph>One</paragraph>
	<paragraph>Two</paragraph>
	<paragraph>Three</paragraph>
</items>

To View:

<div class="items">
	<div class="highlighted">
		<p>One</p>
		<p>Two</p>
	</div>
	<div>
		<p>Three</p>
	</div>
</div>

Known issues

There is a problem when you try to put $text nodes directly inside slots. It has been raised and is tracked in #11149.

The workaround for now is to use elementToElement() conversion helper to create multiple elements instead of elementToStructure().

DowncastDispatcher API changes

Changes to the flow of events in the downcast pipeline

This change was driven by two main factors:

• Converters using the new slots API will trigger events in a recursive way while in any other cases DowncastDispatcher will fire events in a flat structure, one by one,
• The opposite UpcastDispatcher triggers events in the recursive way for all converters so we aim to unify both of them with this change.

You can read details about it in #10376

Set of new methods used to control events flow

With this change converters can ask DowncastDispatcher to start converting children or attributes using the conversionApi's convertItem(), convertChildren() and convertAttributes() methods.

Thanks to this converters can control the flow of events.

Implications of this change

In most cases this should be transparent to developers but it may have some impact if you were cancelling events in this flow.

Long story short, do not use evt.stop() inside conversion event listeners unless you know what you are doing.

editor.conversion.for( 'dataDowncast' ).add( dispatcher => {
	return dispatcher.on( `attribute:xyz`, ( evt, data, conversionApi ) => {
		...

		evt.stop(); // ❌  DO NOT CANCEL EVENTS!
	} );
} );

Change convertInsert() to convert()

The aim was to make this API much cleaner and the convert() method can now convert markers as well, you don’t have to trigger it separately. It’s mostly used internally in our code so we won’t go into details about this one.

New error conversion-model-consumable-not-consumed

We’ve been seeing a numerous pesky issues caused by the fact that some elements in the converter were not consumed. This was very hard to debug and was causing other, unrelated converters to accidentally consume these previously unconsumed elements.

I has been discussed multiple times in the past, mainly in #3818 and we finally introduced conversion-model-consumable-not-consumed error in #10377 which you can check for more info.

❌ What to avoid (notice no consumable.consume in the converter):

editor.conversion.for( 'dataDowncast' ).add( dispatcher => {
	return dispatcher.on( `attribute:xyz`, ( evt, data, conversionApi ) => {
		const { item, attributeNewValue } = data;
		const { mapper, writer } = conversionApi;

		if ( !conversionApi.consumable.test( item, evt.name ) ) {
			return;
		}
		
		const viewElement = mapper.toViewElement( item );

		writer.setStyle( 'xyz', attributeNewValue, viewElement );
	} );
} );

✅  What to to instead:

editor.conversion.for( 'dataDowncast' ).add( dispatcher => {
	return dispatcher.on( `attribute:xyz`, ( evt, data, conversionApi ) => {
		const { item, attributeNewValue } = data;
		const { mapper, writer } = conversionApi;
		
		// ✅ Consume item immediately when you check if consumable.
		if ( !conversionApi.consumable.consume( item, evt.name ) ) {
			return;
		}
		
		const viewElement = mapper.toViewElement( item );

		writer.setStyle( 'xyz', attributeNewValue, viewElement );
	} );
} );

--- OR ---:

editor.conversion.for( 'dataDowncast' ).add( dispatcher => {
	return dispatcher.on( `attribute:xyz`, ( evt, data, conversionApi ) => {
		const { item, attributeNewValue } = data;
		const { mapper, writer } = conversionApi;
		
		if ( !conversionApi.consumable.test( item, evt.name ) ) {
			return;
		}
		
		const viewElement = mapper.toViewElement( item );

		writer.setStyle( 'xyz', attributeNewValue, viewElement );

		// ✅ Consume item afterwards.
		conversionApi.consumable.consume( item, evt.name );
	} );
} );

EditingController API changes

New methods reconvertItem() and reconvertMarker()

For many reasons (very nicely explained in #10659) we introduced these two new methods.

We need them mainly if something changed outside of the model or in a way that you need to manually trigger the reconversion.

It’s usually used in very complex cases so there’s very little change you may be affected by this change.