[defer-hydration] Initial draft

[justinfagnani]

No description provided.

[matthewp]

Can you explain why this is (why they shouldn't automatically hydrate)? Is there an example where doing so would create unwanted results?

[Westbrook]

I could see this fitting into a "when work gets done" topic of conversation by allowing islands to be linked generally by component name, while still being able to disambiguate on which of those islands are needed at any one time. Much like Astrobuild leverages the :visible modifier to control when the code for a component is loaded, this could allow other instances of the component on the same page the ability upgrade with the initially visible instance, and then wait to hydrate until they, too, are in view.

I think this sort of laziness deserves to be revisited more thoroughly a the browser spec level. This seems roughly in line with the a proposal around lazy definitions: WICG/webcomponents#782 and could arguably fit into a larger conversation around the browser allowing for lazy upgrades, which would be somewhat similar to what this protocol looks to support.

Regardless, if you're SSR'ing the entirety of an element that may not come into view, the ability to control when it reconciles that delivered SSR render with the client-side render sounds like a good idea.

[matthewp]

Sorry for just now getting back to this, but why would you want to delay rendering a component? The reason for those modifiers are to:

1. Deprioritize loading of the code so that more important assets can be loaded over network.
2. Deprioritize executing the component (and its dependencies) JS so that more important assets can use the CPU instead.

The reason is not to delay rendering. Rendering should be relatively cheap, especially if the component was prerendered, there's no work to do but set up the event listeners.

It is, of course, possible that some components are going to be more expensive than others, but that's reason for those particular components to have their own API to delay rendering, not to make it a standard that is used arbitrarily.

The goal is to make the page as interactive as possible. Delaying rendering without good reason should be avoided.

[justinfagnani]

Again, the main reason is to enable top-down initialization, even for spontaneous first renders. We need our components to initialize in the same order whether CSR'ed or SSR'ed, independent of definition order.

[matthewp]

That's a different use case (being discussed below). The island orchestration use case is another, it's why Eleventy/webc has adopted this for example.

[matthewp]

I guess this is maybe a partial answer for (1)? Should these be separate points, if so?

[matthewp]

The more I think about it, I don't think I agree with this point (2), cc @Westbrook. I would expect that in most cases an element's initial data has been serialized as attributes and doesn't need the parent to pass that information. The exception would be data that isn't serialized, like complex objects, but I don't think that's really the norm.

[Westbrook]

The https://github.com/w3c/webcomponents-cg has a breakout session to discuss Community Protocols, that could certainly include this, at the beginning of next month. Would you be interested in joining the call? Would be great to get your thoughts on this, and many other of the topics we're discussing right now "in person"!

[matthewp]

Yeah I could probably attend that.

Just for clarity, I'm not against this proposal, just trying to understand the scenarios in which it is needed. The text could help by clarifying that a bit more.

My thinking is that a typical usage would not need this because:

1. Static imports take care of making sure children are hydrated first.
2. Even if hydration happens out-of-order state should be serialized in attributes.

[Westbrook]

@justinfagnani this looks like a great start.

What are you thoughts on this possibly being paired with a conversation at the spec level for revisiting lazy definitions, lazy upgrading, etc.?

Also, @liamdebeasi I wonder how you might see something like this aligning with the lazy loading strategy that Stencil uses. Feel free to ring in any of your teammates that would be better fit to the conversation, if you'd like.

[Westbrook]

I could see this fitting into a "when work gets done" topic of conversation by allowing islands to be linked generally by component name, while still being able to disambiguate on which of those islands are needed at any one time. Much like Astrobuild leverages the :visible modifier to control when the code for a component is loaded, this could allow other instances of the component on the same page the ability upgrade with the initially visible instance, and then wait to hydrate until they, too, are in view.

I think this sort of laziness deserves to be revisited more thoroughly a the browser spec level. This seems roughly in line with the a proposal around lazy definitions: WICG/webcomponents#782 and could arguably fit into a larger conversation around the browser allowing for lazy upgrades, which would be somewhat similar to what this protocol looks to support.

Regardless, if you're SSR'ing the entirety of an element that may not come into view, the ability to control when it reconciles that delivered SSR render with the client-side render sounds like a good idea.

[Westbrook]

visibility is such a hot/important topic for upgrade/hydration that we should probably include it in line with such as interacting with an element.

[thescientist13]

A couple minor grammatical point(s)

change from

The defer-hydration protocol is design to allow controler over hydration to solve two related problems:

to

The defer-hydration protocol is design designed to allow controler control over hydration to solve two related problems:

[splitinfinities]

Curious on folks perspective around defer-hydration as compared to something akin to loading="lazy"

If the attribute was, say, hydration, with options like:

• defer: Hydrates based on internal behavior
• lazy: Hydrates based on when in viewport
• eager: Default behavior

Should the attribute control both the intended behavior and the current state? Some tools use :visible, Stencil uses the hydrated class for hydrated state.

Decoupling this and setting a convention around the intended behavior and leaving how state is communicated up to the tools might be helpful.

[daKmoR]

an additional behavior that might be nice to have is hydration="never". For example if you have purely visual web components.

[matthewp]

I saw that 11ty had implemented this proposal so I wanted to reiterate my questions. What is missing from this proposal is a concrete use-case. What is the use-case where a component should delay its hydration because a parent told it to? What is the parent needing to do to the component before it hydrates?

A good use-case would be something like a special type of component that is tightly coupled to its parent. Point (2) specifies data dependencies, but that is not what I would call a specific use-case.

[JosefJezek]

https://www.zachleat.com/web/defer-hydration/

[justinfagnani]

The biggest motivation for this proposal is to ensure top-down initialization of SSR'ed elements.

In client-side code, shadow children are necessarily created after their parents. So they will have received properties from their parents and can assume that they can fire events that will be handled by their parents.

With SSR'ed HTML and shadow roots elements are already created and will initialize in definition order. This can cause three major problems:

1. A polynomial amount of initialization work. If a child initializes before its parent, it will use default values for properties and render. If a parent then initializes, the parent will use default value for its properties, then write properties into the child, causing the child to render again. If a grandparent then initializes it will do the same, causing the the parent and child to render again.

• 1a: Aside from extra work, mismatches between SSR'ed state and client state are sometimes very bad for hydration, like if a certain hydration pass expects to be given the same data as used in SSR as an optimization.

2. Event listening and dispatch can happen out-of-order causing missed events. defer-hydration lets components wait until their parents are ready and have had a chance to add listeners before dispatching events

In addition to those problems that defer-hydration solves, it also creates an interoperable signal for islands-type architectures to use to trigger components to hydrate by removing the attribute on visibility, interaction, etc,.

[matthewp]

Thanks! I think I see 2 use-cases here. 1 is a tight coupling between components that are not serialized in such a way that they could be hydrated independently. The second is the island-control use-case. I'd like to skip the island idea for now and focus on the first idea because it's the one I understand the least.

Let's put this to real-code. Let's say you have this:

<parent-1>
  <template shadowroot="open">
    <child-1></child-1>
  </template>
</parent-1>

Correct me if I'm wrong, but I think you are saying that <child-1> will render first with its defaults and then again when <parent-1> hydrates with possibly different properties being passed down to <child-1>. And the same would occur again if there were grandparents, etc.

If I'm understanding this correctly, then this can be fixed by serializing <child-1> with the properties (attributes) it needs:

<parent-1>
  <template shadowroot="open">
    <child-1 first-name="Matthew"></child-1>
  </template>
</parent-1>

So my next question is why wouldn't you do it this way? I think I'm missing something, and there's a reason why you can't just serialize.

[kevinpschaaf]

So my next question is why wouldn't you do it this way?

This is indeed a partial solution, but it only works with easily serializable primitive data for e.g. leaf widgets, and breaks down with more complex application data:

• Not all data passed from parent to child is (easily) serializable, in the case of JS property-valued items (structured data, classes, functions, etc.)
• Even if you came up with a serialization method for all data (for now, just assume JSON or similar), serializing real application data that is prop-drilled down the tree would be wildly inefficient (grandparent gets {a: {b: {c: 5}}} serialized to it, parent gets {b: {c: 5}}, child gets {c: 5} etc... with polynomial fanout down the tree)
• Deserializing e.g. JSON necessarily means creating new object instances; thus two children in the tree having been passed the same object will end up with separate object instances, which may break assumptions in the application.

This protocol is attempting to help provide a robust SSR capability for Web Components in the general case. If a given SSR implementation knew all of the data for a given WC instance was serializable to attributes, it could opt to let the instance eagerly hydrate and not use defer-hydration.

Thus, this protocol is addressing cases where that's not possible/practical.

[matthewp]

Thanks! So the use-case is passing complex types.

Are there examples of generic components that would need complex types? Are there examples of native elements that need complex types?

My current thinking is that this use-case is for tightly coupled "application logic" components. These types of components are contained to within a single codebase. Perhaps cross-team within a company. But not a generic component that you might distribute to just anyone.

With that being the case I don't think a community standard is necessary here. If <child-1> requires a complex property to be passed to it, it can define its own API for it to wait.

---

After thinking about it more, my general take is that delaying hydration is bad and should be avoided when possible. Doing something like this within your own application because you value the developer convenience of passing complex objects is completely fine, but it's not something we should endorse.

[dgp1130]

As I've been playing around with this proposal, one thing which has bothered me is the fact that you need to put defer-hydration fully down the render tree for every custom element. This can be tricky to manage and you end up with:

<my-outer-component defer-hydration>
  <template shadowroot="open">
    <!-- I *MUST* put `defer-hydration` here too, or else it will hydrate immediately. -->
    <my-inner-component defer-hydration></my-inner-component>
  </template>
</my-outer-component>

Since defer-hydration is not the default, it means you have to write it for every element. It is very easy to forget to add this to an element, and it's difficult to validate. Particularly for the island use case, you'd want everything under the island to be deferred, and then when that island is hydrated, everything underneath it should be hydrated as well.

I couldn't think of a good way to do this, but I recently found out about custom state pseudo-classes which almost implement this. The key thing about pseudo-classes is that they inherit down the tree just like CSS does. This means that a component at the bottom of the render tree can detect what its parent's state is. If we define it as custom state like :--defer-hydration, this means we can do:

<my-outer-component defer-hydration>
  <template shadowroot="open">
    <!-- No `defer-hydration` attribute here! -->
    <my-inner-component></my-inner-component>
  </template>
</my-outer-component>

class MyOuterComponent extends HTMLElement {
  constructor() {
    super();

    // Set the custom state based on the `defer-hydration` attribute.
    const internals = this.attachInternals();
    if (this.hasAttribute('defer-hydration')) internals.states.add('--defer-hydration');
  }
}
customElements.define('my-outer-component', MyOuterComponent);

class MyInnerComponent extends HTMLElement {
  constructor() {
    super();

    // Only hydrate if the custom state is not set.
    const defer = this.matches(':host-context(:--defer-hydration)');
    if (!defer) this.hydrate();
  }

  // ...
}
customElements.define('my-inner-component', MyInnerComponent);

And the inner component is able to defer based on the parent's :--defer-hydration state! I'd love for this to work, but there are a couple issues today:

1. Custom state pseudo classes are only supported in Chrome and not a fully supported standard.
2. this.matches(':host-context(...)') always returns false AFAICT. This does work if you use it in a real stylesheet to style something.
3. Currently the only API I'm aware of to set state is to use this.attachInternals().states.add('--defer-hydration'). That requires the top-level element to execute and upgrade first in order to set that state based on its attribute before any descendent nodes upgrade, which can be difficult to enforce.
   • A way to set this state declaratively in HTML is probably required. Something like <my-outer-component :--defer-hydration /> would be needed to enable the state without hydrating my-outer-component.
4. Currently we need attributeChangedCallback() to synchronously detect a change to defer-hydration and hydrate. I'm not aware of such a hook for detecting CSS. I believe you can use MutationObserver to detect some CSS changes, but it is async and likely not sufficient. Some kind of customStateChangedCallback() would likely be needed.
   • To add to this complexity, ordering of the customStateChangedCallback() is important. It would probably need to flow top-down so each element can choose whether to hydrate its children (do nothing) or keep deferring them (set --defer-hydration until some other user interaction).
5. Custom states can only be set via ElementInternals which is designed to be private to the custom element, meaning you can't imperatively remove :--defer-hydration from another element like you can with the defer-hydration attribute as defined already.
   • Maybe this could be solved via a community protocol, but I'm not 100% on what that would look like.

I'm not sure if this is really a good direction for defer-hydration, but I thought it was an interesting approach to not having to manually repeat defer-hydration all the way down the tree. This might be a bit of a hack of the custom states use case and maybe at that point we should just have a proper web spec for deferring hydration. Food for thought.

Edit: In retrospect, you can do a lot of this with plain :host-context([defer-attribute]). In fact, doing that solves problems 1., 3., and 5. Number 2. is still an issue but mostly the same scope of problem.

Number 4. is actually a bit worse, since there's no clear indicator of when the relevant selector might change for :host-context([defer-attribute]). We'd need an API like Element.prototype.matches, except as an event listener for when the result changes.

class MyComponent extends HTMLElement {
  constructor() {
    super();

    if (!this.matches(':host-context([defer-hydration])')) {
      this.hydrate();
    } else {
      this.onMatchChange(':host-context([defer-hydration])', () => {
        this.hydrate();
      });
    }
  }

  // ...
}

customElements.define('my-component', MyComponent);

To hydrate effectively, that API still needs to call back synchronously, and given that any DOM manipulation could result in changing match state, that means a lot of APIs could now synchronously cause an element to hydrate, which might be difficult to manage from an implementation perspective.

[kevinpschaaf]

But not a generic component that you might distribute to just anyone.

@matthewp I'd challenge you to imagine a future where <data-tree> and <awesome-spreadsheet> and <code-editor> can be published as reusable components, that can be dropped into any application, and enjoy all the same benefits of server-rendering that buttons will. In these cases, an API with complex objects is not simply a "developer convenience," it's a practicality of making components that require configuration that exceed the limits of primitive data types (which itself is a completely arbitrary constraint).

Another angle here is that, even with "application-specific" components, there is immense value in the flexibility that web components provide in the ability to incrementally migrate a given component or codebase from one WC base class to another; protocols like this that define any extra interface points above the standard spec'ed interface ensure that's also possible.

If the extent of your vision for web components stops at <my-button label="Simple">, like I said you won't be a customer of this protocol, and that's fine. But it needn't stop defining a protocol for those who do want to make more complex interoperable reusable components.

[kevinpschaaf]

@dgp1130 That would be a pretty clever approach, but unfortunately besides the defacto limitations you mentioned with matches() and detecting changes, :host-context is also a somewhat disputed part of the V1 web components spec (see w3c/csswg-drafts#1914) and does not have good cross-browser support. Also, in our real-world usage of defer-hydration so far, it's important to be able to incrementally remove defer-hydration element-by-element down the tree rather than once and have it inherit down, since some great-great-grandparent can't reliably know when it's safe to assume hydration down the tree is complete in order to remove the state.

[matthewp]

@kevinpschaaf Thanks for some more specific use-cases. I can imagine these types of components as I'm the author of one myself. Mine does not support server rendering at the moment. If it did I can imagine I would infer if I needed to render defaults.

You can do this easily with declarative shadow DOM by checking if this.shadowRoot exists. If it does then that means it was server rendered and you can skip rendering its defaults.

If the extent of your vision for web components stops at <my-button label="Simple">, like I said you won't be a customer of this protocol, and that's fine. But it needn't stop defining a protocol for those who do want to make more complex interoperable reusable components.

This protocol is recommending this approach for all components. From the proposal:

"1. Interoperable incremental hydration across web components. Components should not automatically hydrate upon being defined, but wait for a signal from their parent or other coordinator."

This is the source of our disagreements. Components absolutely should hydrate upon being by defined. Only in very special circumstances should they not do so.

The fact that custom elements can hydrate independent of a tree is the biggest benefit of web components. Proposing that they not do so, and instead way for some root element to first hydrate makes them more like a React tree. We should not make exceptional scenarios the default.

Keep in mind that Eleventy/webc has implemented this proposal, so they are conforming to what is in my opinion a bad default, right now.

[matthewp]

My biggest issue with this proposal is that it is a blanket recommendation that all components defer to the wishes of a parent component on when to hydrate. A element should always decide for itself when to hydrate.

This is a case-by-case basis. Some native elements have similar constraints, but they always define an API in which they are in charge and the user only provides a hint. For example <img loading=lazy> as defined on MDN:

Defers loading the image until it reaches a calculated distance from the viewport, as defined by the browser.

Note the important part as defined by the browser. That is, the browser defines when to take action, it doesn't wait for the application developer to tell it that it's time.

[kevinpschaaf]

You can do this easily with declarative shadow DOM by checking if this.shadowRoot exists. If it does then that means it was server rendered and you can skip rendering its defaults.

I don't follow. this.shadowRoot existing only tells you that it was server rendered. It does not tell you when and if the host is done configuring the element with any properties that may be coming. The existence of defer-hydration is an explicit signal from the host that it will provide data, and its removal is an explicit signal that it has completed providing data.

This protocol is recommending this approach for all components.... This is the source of our disagreements.

My biggest issue with this proposal is that it is a blanket recommendation that all components defer to the wishes of a parent component on when to hydrate.

I see. In general it should be possible for defer-hydration to be an opt-in protocol, but I think I agree that there's a possible ambiguity in whether a rendering context knows whether it should set defer-hydration on an element it is rendering, given that the element may not implement defer-hydration AND still be configurable purely from attributes. Perhaps this argues for only setting defer-hydration when a rendering context detects it is setting a non-serializable JS value. Or maybe it should be a thing that's set via an ElementRenderer interface for a given WC class (which is specific to the element being rendered) rather than from the rendering context... need to think more about whether that works.

A element should always decide for itself when to hydrate.

Sorry, but I don't think this follows. If we can agree that an element shouldn't hydrate until it is fully configured (for performance reasons, amongst others discussed), this is never something a component can decide for itself. A component cannot know when or if a host will or has configured it, given it's just as valid for a host to do nothing to an element as to set a bunch of elements/properties. In the general case, this is only knowable by the host.

[dgp1130]

@dgp1130 That would be a pretty clever approach, but unfortunately besides the defacto limitations you mentioned with matches() and detecting changes, :host-context is also a somewhat disputed part of the V1 web components spec (see w3c/csswg-drafts#1914) and does not have good cross-browser support.

@kevinpschaaf, I wasn't aware of implementation issues with :host-context(). I don't think this approach specifically relies on that though, you can use any value which inherits down the render tree such as a CSS variable instead, though setting it in HTML does get a bit weird:

<my-component style="--defer-hydration: true">
  <my-child-component></my-child-component>
</my-component>

class MyChildComponent extends HTMLElement {
  constructor() {
    super();

    const defer = getComputedStyle(this).getPropertyValue('--defer-hydration')!.trim() === 'true';
    if (!defer) this.hydrate();
  }
}

customElements.define('my-child-component', MyChildComponent);

Definitely a hack and doesn't help with listening for changes to the --defer-hydration variable, but it at least doesn't require :host-context(). Maybe cssVariableChangedCallback() is more amenable to browsers than a synchronous Element.prototype.onMatchChange()? I dunno, probably still a dead end.

Also, in our real-world usage of defer-hydration so far, it's important to be able to incrementally remove defer-hydration element-by-element down the tree rather than once and have it inherit down, since some great-great-grandparent can't reliably know when it's safe to assume hydration down the tree is complete in order to remove the state.

I think you could still support this use case by actually writing defer-hydration anywhere you wanted more fine-grained control. Your tree could look like:

<my-parent-component defer-hydration>
  <!-- Loads as soon as parent's `defer-hydration` is removed. -->
  <my-component></my-component>

  <!-- Still deferred when parent's defer-hydration is removed.
         `my-parent-component` can manually remove this `defer-hydration` when ready. -->
  <my-lazy-component defer-hydration></my-lazy-component>
</my-parent-component>

This does require knowing which components will be hydrated lazily during render time, but I guess we're already assuming that for defer-hydration in general, so I don't think that's too much of an ask.

Alternatively if hydration happens top down, when my-parent-component hydrates it could add defer-hydration to my-lazy-component to prevent it from hydrating. But that would mean that my-lazy-component has its defer-hydration state briefly cleared (when my-parent-component's defer-hydration attribute is removed), but then later re-set (by my-parent-component on hydration), and this invalid state needs to avoid triggering my-lazy-component's hydration. That feels way too nuanced to me and I think it's probably better to require defer-hydration to be explicitly authored for any subtree which needs to be deferred longer than its parent, which I think is a reasonable requirement.

[kevinpschaaf]

Definitely a hack and doesn't help with listening for changes to the --defer-hydration variable

@dgp1130 Yeah, something like a StyleObserver would be needed to do this robustly, which doesn't exist yet.

Alternatively if hydration happens top down, when my-parent-component hydrates it could add defer-hydration to my-lazy-component to prevent it from hydrating.

An issue here is that defer-hydration must be present prior to upgrade to have an effect. In a typical a renders b renders c tree, a imports b, and b imports c, meaning registration order goes c then b then a due to side-effect registration happening when dependent modules load, and this goes bottom-up the module graph. Hence a won't have a chance to set defer-hydration on b before b upgrades.

Thus in the general case, I think it's always going to be safest to apply defer-hydration on the server when needed.

[dgp1130]

Alternatively if hydration happens top down, when my-parent-component hydrates it could add defer-hydration to my-lazy-component to prevent it from hydrating.

An issue here is that defer-hydration must be present prior to upgrade to have an effect. In a typical a renders b renders c tree, a imports b, and b imports c, meaning registration order goes c then b then a due to side-effect registration happening when dependent modules load, and this goes bottom-up the module graph. Hence a won't have a chance to set defer-hydration on b before b upgrades.

Thus in the general case, I think it's always going to be safest to apply defer-hydration on the server when needed.

Yes, I've talked before about how import order naturally causes bottom-up hydration and this can make certain tasks more difficult. However, I think defer-hydration is the solution to that problem by allowing top-down hydration when it is beneficial to do so. In your example with my approach of having defer-hydration apply transitively down the entire render tree, then putting defer-hydration on a will prevent hydration on b and c, regardless of import ordering. Even if c upgrades first, it would still see that a transitive parent (a) has --defer-hydration set and skip its hydration until that property is removed.

This does imply that a must be deferred in order to defer any of its children imperatively, but I don't think that's strictly true, given that you don't have to define defer-hydration on a custom element specifically, you could put it on any element.

<my-a>
  <template shadowroot="open">
    <div defer-hydration>
      <my-b>
        <template shadowroot="open">
          <my-c></my-c>
        </template>
      </my-b>
    </div>

    <div><!--  Other stuff in my-a --></div>
  </template>
</my-a>

In this example, my-a will hydrate eagerly but my-b and my-c are deferred, even if they upgrade before my-a. Once my-a removes the defer-hydration attribute from the div tag, only then will my-b and my-c hydrate (possibly in a top-down fashion if we can get listening to pseudo selector / CSS variable / arbitrary style change to apply top-down).

I don't mean to distract too much from the overall defer-hydration proposal, I'm happy to make a separate issue to discuss whether it should apply transitively or not. I just haven't seen a convincing argument against it other than the fact that listening to hydration changes isn't really possible today (which feels solvable to me in the browser spec, but I'm not an expert in that aspect).

[matthewp]

I think merging this is a huge mistake. I have disagreed with some of the proposals in this repo but this is the only one that I feel is actually harmful. As stated above, this proposals goal is to abstract away the difference between client-side rendering and hydrated rendering. It does so by making hydrated rendering slower.

It's perfectly fine for a framework to chose this trade-off, but making it a recommendation for the community to also do so, is harmful.

[Westbrook]

As a reminder the "Proposal" status level that is suggested as the next step for this conversation is outlined as follows:

Proposal "Proposal" status applies to just about anything that community members are interested in putting some thought into. While an issue submitted to this repo can help generate initial ideas on the protocol or space of interest and clarify the information needed to kick off a more fruitful conversation, a PR will serve to make known that you are interested in support to drive the protocol in question forward. Having an initial explainer included in this PR, while adding the protocol to the "Proposals" table shown above, will prepare the community to both communicate about and contribute to the development of the protocol asynchronously.

For anyone with continuing concerns about the current shape of the proposal, leveraging issues or PRs for modifications is a great way to follow up on those, or if your concern is even deeper, then you might find submitting an alternate proposal that you felt addressed the concepts herein in a more palatable manner productive to your cause.

[justinfagnani]

@matthewp

It's perfectly fine for a framework to chose this trade-off, but making it a recommendation for the community to also do so, is harmful.

For frameworks to choose this trade-off they need a specification to interop over. This is that specification. It's entirely opt-in, and once again the defer-hydration attribute is only a signal. Elements can do with it what they wish.

We've determined on the Lit project that this feature is required for proper hydration, and will use this feature. By publishing a protocol proposal here we are making it possible for others to interop with us in a formalized way. That is the point of community protocols.

I strongly disagree with your assessment of this protocol, but regardless of that there is room in community protocols for protocols that not everyone agrees with. There's even room for competing protocols. As long as they can be named, referenced, and are specified enough for multiple independent and compatible implementations this repository is serving a good purpose.

[matthewp]

By publishing a protocol proposal here we are making it possible for others to interop with us in a formalized way. That is the point of community protocols.

That's not the point, if that was the case then everything would get merged. If this is truly the case then I have at least 1 proposal of my own that I'll be getting merged soon.

But I don't think that you think this is the point of the repository, as there are no non-Lit involved proposals that have ever been merged.

[justinfagnani]

@matthewp

That's not the point, if that was the case then everything would get merged.

I think there should be a high bar for just blocking a proposal, especially because say that there needs to be two independent implementations to move to "accepted".

You seem to want a veto here, but we haven't established anything like that in the process. Must we satisfy your objections to even make a Proposal even though we've made it very clear our rationale and the problems we already solve with this protocol?

If this is truly the case then I have at least 1 proposal of my own that I'll be getting merged soon.

Given your tone in this conversation, would you actually be doing this in good faith? Do you use the protocol today? Are you truly looking to interop with other components and libraries? Would you show up to WCCG meetings to discuss it? Because if so, then make the proposal! If you're just trying to gum up the works I think your energy would be better spent elsewhere.

[nolanlawson]

I haven't fully grokked the details of this proposal, but we (LWC) will probably be evaluating whether to implement it in the next couple months or so.

I'd be really interested to understand better the concerns around this proposal, and I'm also still a little unclear about what would "count" as a framework-interoperable implementation (e.g. see my question on nitty-gritty details).

Maybe this would be worth a breakout session for the WCCG?

[mattlucock]

The other proposals/drafts that are currently being developed (context, task, slottables) all relate to the internal implementation of a component. I think this is part of what makes them so interoperable. I think that in this context, we need interoperability to not just mean interoperability with components that implement those protocols, but also interoperability with components that don't, which in general is most components. A component that doesn't implement a protocol shouldn't care about those that do (a property which I think this proposal maintains) and a component that does shouldn't care about those that don't (a property I don't think this proposal maintains).

This proposal seems fundamentally different to the other proposals in that it seeks to apply behavior to an instance of a component from the outside. Only some web components will be written to support defer-hydration, and so in general, only some of a parent's children will support it. With the proposal as currently written, we will inevitably ask children to defer hydration that do not support it, since by default they don't. As such, it seems to me that when we ask a child to defer hydration, we should assume by default that the child will not honor that request, which makes me question exactly what the practical value is of making the request in the first place.

Much has been said about this being opt-in, but I'm not sure it's opt-in in quite the same way as the other protocols. It seems to me that if we seek to involve components that have not opted-in, we we will draw incorrect conclusions about the behavior of those components. But how do we ensure we only seek to involve components that have opted in? One idea I have is for components to have a SUPPORTS_DEFERRED_HYDRATION = true property, but I don't think I like this; it's a runtime property that's completely irrelevant at runtime.

But even so, if only some children support deferring hydration, and the others don't, then I feel like:

• defer-hydration cannot, in general, be relied upon to facilitate a protocol where parents listen for events from their children, unless the protocol stipulates that the children must honor defer-hydration.
• defer-hydration cannot, in general, be relied upon to prevent the class of potential bugs that it seeks to prevent.

So I'm unsure what the true benefit of defer-hydration can be in practice.

[mattlucock]

I overlooked this in my previous comment, but components have to opt-in to hydration in the first place, do they not? This might be a silly question, but how do we actually know whether we can safely SSR any given component? Is SSR interoperable? (And if not, how can defer-hydration be?)

[matthewp]

@mattlucock All good points that I agree with. This being a proposal that component behavior is added from the outside is indeed unlike the others.

As far as the opt-in part, I think that term is being used as a sort of cover for what is really going on. It's not actually opt-in at the component level, you can go read Lit source code and see that they are opting all Lit components in, full stop. See here. I think their hope is that they can convince enough other frameworks to do the same, then essentially a big chunk of custom elements in the wild will have been opted-in, even though the author did not make that decision on their own. It doesn't matter to them that some others will not have it, they are betting on the numbers being in their favor.

[mattlucock]

I think I was a bit confused when reading the proposal a few weeks ago—although I think that's because the proposal is confusing. Reading it again, it's not apparent that a parent 'asks' its children to defer hydration like I thought; it more seems like a component would determine for itself that it should defer hydration when being SSR-d. But this actually just creates a different variation of the same problem: how does a child know whether its parent participates in the protocol and will remove the defer-hydration attribute from it? If the parent never removes the attribute, the child never hydrates.

@matthewp It makes complete sense to me that a framework for SSR-ing web components could opt all components in to something like this by default; this thing would basically just be an implementation detail of components built using the framework. But as I was getting at, it feels like this only makes sense in the context of a particular framework, and it feels like this fundamentally cannot be made interoperable in any way that makes sense.

---

I am concerned by the approach being taken by this proposal, which appears to be "we've implemented this in Lit, and now we're making this a community protocol so anyone can choose to interoperate with us if they want to". Not only does it presuppose that Lit's design decisions are the right ones and should determine the design of components broadly, but it also presupposes that these things can and should be interoperated with.

We've determined on the Lit project that this feature is required for proper hydration, and will use this feature. By publishing a protocol proposal here we are making it possible for others to interop with us in a formalized way. That is the point of community protocols.

I think this is fundamentally the wrong approach. We can't build community consensus for design decisions if the design decisions have, apparently, already been made, and aren't really up for debate. 'Formalizing the design of Lit' is not what this group should be trying to achieve.

[nolanlawson]

I'm actually glad that @justinfagnani put the effort into documenting how Lit does things, so that other frameworks might be able to interoperate with Lit. This is kind of the whole goal of web components.

Thinking more about this proposal, though, I feel like it may be putting the cart before the horse – before figuring out how 2 WC frameworks should hydrate together, we need to figure out how they'll SSR together. E.g. if LWC wanted to implement support for SSR'ing Lit components and hydrating with them, it's not enough to use defer-hydration correctly – we'd also have to figure out some generic way to SSR a WC (and ideally use some kind of standardized abstraction layer and not just directly import @lit-labs/ssr).

Again, I'm not against this proposal, but I'd like to see some brave WC framework author figure out how to SSR/hydrate Lit components, and then it'd be clearer to me how this is supposed to work in practice.

Alternatively: if this proposal could be distilled down into something non-hydration related (e.g. defer-work or defer-rendering, etc.), then maybe it could serve as the low-level primitive that the more complex system could leverage.

[mattlucock]

@nolanlawson The point I was trying to make is that "the whole goal of web components" is not, and cannot be, to "interoperate with Lit". I think I may be over-emphasizing this concern, but this framing and thinking is really unproductive for what is supposed to be a community initiative, and I also think it's backwards: it's not that web components should interoperate with Lit, but that Lit should interoperate with web components.

I think I generally agree with the rest of your sentiment, but I am a bit pessimistic about there being a distilled form of this that can be truly interoperable. There is nuance to this, which I mentioned in my previous comment, in that this seeks to apply behavior to a component from the outside, only some components will support it, and if we try and apply it to a component that doesn't support it we will draw incorrect conclusions about that component's behavior.

[nolanlawson]

@mattlucock These are valid concerns. The goal here is definitely interop among multiple WC frameworks – not just Lit. However, Lit might be at the vanguard of figuring out this whole SSR/hydration thing, so I take their proposal seriously when they say that something like defer-hydration is a useful primitive.

If it's not, then by all means, let's build something better! But I respect Lit for coming to this forum to at least reach out to other WC frameworks and try to figure out how to interoperate.

[dgp1130]

One thing I just realized is that attributeChangedCallback actually swallows errors and does not propagate them to whomever called el.removeAttribute('defer-hydration').

If I do:

try {
  el.removeAttribute('defer-hydration');
  console.log('No error detected?');
} catch (err) {
  console.log('Caught error!');
}

On a component which throws an error, I get:

Uncaught Error: Hydration failed. :(
    at ErrorComponent.attributeChangedCallback (script.js:6:11)
    at script.js:14:6
No error detected?

Stackblitz minimal reproduction.

For this proposal, that means it is impossible to detect a hydration error from another component. Is this an intentional design choice or an oversight? I don't see the word "error" in this PR or its discussion.

It is possible to workaround, though it's a bit hacky. We could do something like:

declare global {
  var lastHydrationError?: unknown;
}

class HydratingElement extends HTMLElement {
  hydrate(): void {
    throw new Error('Oh noes!');
  }

  static observedAttributes = ['defer-hyration'];
  attributeChangedCallback(name: string, _oldValue: string | null, newValue: string | null): void {
    if (name !== 'defer-hydration' || newValue !== null) return;

    // Clean up any pre-existing errors, in case they weren't caught.
    delete globalThis.lastHydrationError;

    try {
      this.hydrate();
    } catch (err) {
      // If I `throw` here, no one will catch it, so I can only log the error instead.
      globalThis.lastHydrationError = err;
    }
  }
}
customElements.define('hydrating-element', HydratingElement);

function hydrate(el: Element): void {
  el.removeAttribute('defer-hydration');
  // ^-- Won't `throw` if anything went wrong.

  // Check `lastHydrationError`.
  const error = globalThis.lastHydrationError;

  // Clean up `lastHydrationError` so the next hydration doesn't pick it up.
  delete globalThis.lastHydrationError;

  // Propagate error to consumers.
  throw error;
}

hydrate(document.querySelector('hydrating-element'));
// ^-- Throws as expected.

This is certainly an awkward contract and I'm sure there are better ways to do it, but it's a trivial workaround which is able to propagate hydration errors. Unfortunately this would need to be baked into the protocol. Is it worth having a mechanism to detect hydration errors? Is there a better approach than what I'm describing?

[nolanlawson]

@dgp1130 I think what you're running into is called a "custom element callback reaction" error. It should be catchable with an error listener on the window.

I've found this to be a generic problem any time you're trying to catch an error in connectedCallback, form association callbacks, etc. The best solution I've found is the global window listener.

[dgp1130]

@nolanlawson Thanks for that context and excellent blog post. I can definitely understand why attributeChangedCallback works that way. It does make some amount of sense that el.setAttribute('attr', 'value') shouldn't ever fail. But it is a challenge for this proposal.

Would it be feasible to use the window error event to catch an uncaught attributeChangedCallback error and be confident that was the source of the error? I'm not convinced. MDN states:

The error event is fired on a Window object when a resource failed to load or couldn't be used — for example if a script has an execution error.

So more than just uncaught callback reaction errors will trigger it. Is there a risk of potentially catching errors which did not originate from an attributeChangedCallback handling the defer-hydration removal?

Fortunately attributeChangedCallback is synchronous, so resource load error or script execution error shouldn't be an issue in this context. However, there are three cases I can think of where listening to window error would catch incorrect errors.

First is that component or its dependencies may call window.dispatchEvent('error') at any time. I can't say I've seen this in practice too much but I could see some scripts essentially throwing an async error via this mechanism.

Second is composed hydration. Take the following example of hydration "call stack":

<outer-component>
  <inner-component defer-hydration>
    <error-component defer-hydration></error-component>
  </inner-component>
</outer-component>

In this case, we're using defer-hydration to create top-down hydration timing. outer-component hydration triggers inner-component hydration which triggers error-component hydration.

error-component throws an error on hydration in attributeChangedCallback. Both outer-component and inner-component have listeners for window error, but they cannot distinguish whether the error is coming from a direct descendent. In a "real" call stack, an error from error-component could be caught and ignored by inner-component, preventing outer-component from ever seeing it. But with this event listener, both inner-component and outer-component observe the error. There is no way for outer-component to know that the error was caught and handled by inner-component.

Example Stackblitz

Third is catching errors from setAttribute or connectedCallback invocations unrelated to hydration. Take for example:

class MyComponent extends HTMLElement {
  static observedAttributes = ['defer-hydration', 'attr'];

  attributeChangedCallback(name, _oldValue, newValue) {
    if (name === 'defer-hydration' && newValue === null) {
      this.onHydration();
    } else {
      throw new Error('Unknown attribute'); // Caught by `window` `error` callback.
    }
  }

  onHydration() {
    this.setAttribute('attr', '');
    console.log('Hydration successful!'); // Logs as expected.
  }
}

That setAttribute call can produce a window error event if attr happens to be observed by the component and an error is thrown in attributeChangedCallback. However attributeChangedCallback did not throw in the defer-hydration case, which is the error we'd actually want to catch. The unrelated error when modifying attr is swallowed and shouldn't fail hydration.

Stackblitz demo

One other challenge is that multiple attributeChangedCallback or connectedCallback errors might result in multiple errors caught by window error. I suspect it would be slightly more correct to construct an AggregateError of each caught error. Alternatively, the last error might be the most important because it is likely the one which was actually thrown in attributeChangedCallback. Even that's not perfect however, as it ignores the use case of one component catching and ignoring the error from another component.

Does window error give more information about the events it emits such that we could distinguish some of these cases? My guess is no, but documentation on this seems a little lacking. My intuition is that using window error for hydration errors in this context would be too inconsistent and ambiguous to be generally useful. Would love to hear others' thoughts.

[nolanlawson]

Would it be feasible to use the window error event to catch an uncaught attributeChangedCallback error and be confident that was the source of the error?

If you set the global error listener right before calling setAttribute, and if you check event.error.stack to see if it contains attributeChangedCallback, then you could be reasonably certain. But it'd still be kind of janky.

[dgp1130]

Wouldn't that still fail for the third example I gave? Any error in an attributeChangedCallback will have that method in the stack trace. We'd need to identify that it was ErrorComponent.prototype.attributeChangedCallback specifically. That sounds tricky with minifiers (though potentially possible). But I think it would fail in the case where there are two ErrorComponent classes since they would not be distinguishable. Beyond that you could have two instances of an ErrorComponent, which one we want to catch depends on which one I was hydrating, which I don't think is knowable by looking at stack traces.

[nolanlawson]

@dgp1130 IIUC yeah, if you have an attributeChangedCallback that is itself invoking another setAttribute whose attributeChangedCallback throws an error, then yeah, I don't see how you could reliably distinguish the two errors. It would have to be part of the defer-hydration contract to throw a particular type of error or something.

[Westbrook]

We agreed as per the March monthly WCCG meeting that merging this PR is appropriate, being it targets the "Proposal" status, the lowest possible level. We'll do our best to capture any conversations into issues for follow-up as the protocol moves up the various status levels (or not), so feel free to get your last comments posted over the next week and a half. On the 25th of March this PR will be merged as a "Proposal" protocol.