Structured cloning and transferring

[annevk]

I'm rewriting the way we defined structured cloning in whatwg/html#727 to more closely match ECMAScript.

The tentative decision has been that since some members in TC39 do not care much for structured cloning and transferring we would not try a generic extensible approach for ECMAScript-defined objects and instead define for each individually what happens as part of a giant switch statement.

That means that the SharedArrayBuffer specifics need to be defined in HTML. I suspect this is best to be done after the aforementioned PR lands.

Is there language written for this already someplace that I can study?

[lars-t-hansen]

The last section of the current spec discusses cloning and transfering: https://tc39.github.io/ecmascript_sharedmem/shmem.html#WebBrowserEmbedding.CloneAndTransfer

EDIT: I am removing the prose from the spec and making the spec reference this bug and the whatwg bug. The prose I'm removing is the following:

  <emu-clause id="WebBrowserEmbedding.CloneAndTransfer">
    <h1>Transmitting and Cloning SharedArrayBuffers and Views</h1>

    <emu-note>
      <p> (<a href="https://github.com/tc39/ecmascript_sharedmem/issues/65">Issue 65</a>) 
This will move to the HTML spec. </p>
      <p> (<a href="https://github.com/tc39/ecmascript_sharedmem/issues/39">Issue 39</a>) 
We need to formalize what happens if illegal sharing is attempted, eg, attempting to share
with a ServiceWorker.  See earlier section. </p>
    </emu-note>

    <p> SharedArrayBuffers and TypedArrays that view SharedArrayBuffers can be
transmitted from one worker to another using postMessage() and are handled by
the HTML5 structured clone algorithm as follows.</p>

    <p>When a SharedArrayBuffer input object is cloned in the originating agent the
result is a new SharedArrayBuffer object in the receiving agent that references the
same Shared Data Block as the input object.  The input object must be present in
the transfer map for the clone operation; if not, a standard clone exception will be
thrown.</p>

    <p>When a SharedArrayBuffer object is cloned its Shared Data Block is not
detached (SharedArrayBuffers are never detached).</p>

    <p>A Shared Data Block remains accessible so long as at least one SharedArrayBuffer
object references it in some agent.</p>

    <p>When a TypedArray object that is a view on a SharedArrayBuffer is cloned the
result is a new TypedArray object of the same element type.  The "buffer" property of
the new TypedArray will reference a clone of the input object's "buffer" property.
The input object's "buffer" property must be present in the transfer map for the clone,
or a standard clone exception will be thrown.</p>

  </emu-clause>

[annevk]

Okay, so you only support transfer semantics for these objects, cloning throws, with the caveat that transferring actually entails sharing the underlying data. That will be relatively easy to add to the HTML Standard.

[lars-t-hansen]

Yes, and I think that's sufficient. Just bear in mind that few people have had a chance to test the ergonomics of these APIs yet.

[annevk]

Notes for the eventual PR:

• Add a note under "Transferable objects" that SharedArrayBuffer does not follow the invariant we require from IDL-defined objects (it doesn't detach)
• Update IsTransferable
• Update TransferHelper

[lars-t-hansen]

Detatching can probably be supported if it's important to somebody, since it's a local operation that does not require any coordination among workers, but since the transfer syntax is now used to transfer SABs when they do not detach I don't know what it would look like in practice. Now's a good time to speak up.

[annevk]

Detaching only makes sense when you truly transfer. While it uses the transfer infrastructure, SAB shares, so detaching doesn't quite apply. (That's mostly what I want to point out in the note.)

I don't think detaching is used for anything but transferring so I don't think it needs to be added for SAB.

[annevk]

Now that HTML has rewritten the cloning and transferring operations in terms of ECMAScript, we could add this to the HTML Standard. I think it might be a good idea to do that since it's implemented already and I am willing to do the work. What do you think?

I guess the only problem is that SAB itself is not defined inside ECMAScript yet so we'd have to reference a a separate draft, but that does not seem so bad.

[lars-t-hansen]

Obviously I'm ecstatic to remove things from any specs I'm responsible for, so as far as I'm concerned you should go right ahead. And feedback from people who care about HTML is always useful. Just don't nail anything down too tightly until the shared memory spec reaches stage 3, please.

Also note bug #38, about IDL and DOM, in case you didn't see it.

[annevk]

Can a SharedArrayBuffer ever be detached?

[annevk]

Assuming it cannot be detached, these would be the changes:

• We would change IsTransferable by returning true if O has a [[SharedArrayBufferData]] internal slot.
• We would change TransferHelper to create new SharedArrayBuffer object in targetRealm whose [[SharedArrayBufferByteLength]] internal slot value is the [[SharedArrayBufferByteLength]] internal slot value of input, and whose [[SharedArrayBufferData]] internal slot value is the [[SharedArrayBufferData]] internal slot value of input, if input has a [[SharedArrayBufferData]] internal slot.
• As for the transferable objects section, I think I would add a paragraph at the end explaining SharedArrayBuffer to be a special transferable object in that transferring it does not cause it to be detached and does allow it to be used again. I don't think we'll introduce another object like it so that seems sufficient.

[lars-t-hansen]

@annevk

Can a SharedArrayBuffer ever be detached?

No, it cannot at present be detached. Mark Miller made an argument that there are some interesting use cases for detachment, but I had not planned to act on these for v1. (summoning @erights)

[erights]

The tentative decision has been that since some members in TC39 do not care much for structured cloning and transferring we would not try a generic extensible approach for ECMAScript-defined objects and instead define for each individually what happens as part of a giant switch statement.

Sorry I had not noticed this thread till now. @annevk I do not understand the paragraph I quote above. Can you clarify? Thanks.

[erights]

@lars-t-hansen @annevk Yes, I believe that detachment is a clean way to contain inconsistency following preemptive termination. I do yet not understand how else the SAB proposal plans to handle the preemptive termination of one of the participants.

[annevk]

@erights it's mostly about how StructuredCloneWithTransfer in the HTML Standard is organized. ECMAScript-defined objects are part of a case-switch-like construct whereas IDL-defined objects have a [[Clone]] and can have a [[Transfer]] hook so that each specification can keep their objects self-contained without having to patch the HTML Standard for their cloning and transferring bits.

[lars-t-hansen]

OK, summarizing the discussion from issue #55 regarding detaching SAB (to be written up in the Agents spec):

• Sudden, forced SAB detachment - ie without the cooperation of the agents that are not doing the detaching - is racy and almost certainly deeply unsafe.
• Non-sudden, cooperative SAB detachment may be nice to have but there probably are no strong use cases for it yet; the use case for the sudden, forced detachment was security against becoming confused, but cooperative detachment provides only very limited guarantees in that respect.

(Cooperative SAB detachment requires all the agents that share memory to cooperate, as a first approximation we could say that a SAB could be detached in an agent iff all the other agents that share it are blocked at the event loop; a second approximation would be to add futexWait to the set of blocking points. This will tend to be brittle in practice, is my guess.)

The conclusion, for now, is that SABs will not be detachable.

[lars-t-hansen]

There is a related matter being discussed in #98, where the (tentative) conclusion is that SAB will not be "transfered" by structured cloning, this is generally just a surface syntax matter, but one can say w.postMessage(sab), one does not have to say w.postMessage(sab, [sab]) to send it to w.

I'm going to leave this ticket open for now but it doesn't pertain directly to the ES spec now that we've decided not to allow detaching SABs.

[annevk]

I think this can be closed now.