Allow passing through (Shared)ArrayBuffer

[Jamesernator]

The new proposal for isolated Realms breaks uses relying on internal slots of objects. For example, even with a membrane between two realms one cannot preserve semantics such as myRealmGlobalThis.Set.prototype.get.apply(otherRealmSet, "someKey") (this will throw a TypeError as the membrane objects won't have the right internal slots).

While for the most part, I don't actually think this is a big deal, and I generally think the upsides of isolated realms outweigh this particular downside. Most cases where this pattern would be necessary, can be easily resolved by using the functions of the realm where these objects came from (e.g. otherRealm.Set.prototype.get.apply(otherRealmSet, "someKey") still works).

However, I think one case where this is more of a problem is with the ArrayBuffer and SharedArrayBuffer objects. Both of these objects are designed to be performant and in many cases are designed to be used without memory copy (e.g. transfer steps in browsers). Having to go through a membrane to work with ArrayBuffers, and having to copy them entirely to pass them to host APIs kinda defeats their performance benefits. (And even in the current time, there are new proposals to make such abilities even easier).

This is even worse for SharedArrayBuffer, as if a SharedArrayBuffer lives inside a child Realm it is impossible to share it to other threads, period, as it cannot get out of the realm to be passed to an appropriate host API (e.g. worker.postMessage). This fundamentally breaks the whole capability of the SharedArrayBuffer object and makes it fairly useless within a child realm that lacks host APIs.

As such I'd like to propose, that like functions ArrayBuffer and SharedArrayBuffer are able to pass between realms, except of course, that like functions they are simply given a new identity object. i.e.:

const realm = new Realm();

const buffer = realm.evaluate(`
  const buffer = new Uint8Array(1);
  globalThis.arr = arr;
  arr.buffer;
`);

buffer instanceof ArrayBuffer; // true, recreated in this Realm
const arrOuter = new Uint8Array(buffer);
console.log(arrOuter[0]); // 0
realm.evaluate(`arr[0] = 10`);
console.log(arrOuter[0]); // 10, although the identity object is different, it's still the same memory

[leobalter]

I'm not convinced to explore special treatments in this API to access non-callable objects.

I believe with a fair use case, the Realms API could explore a special method to access buffers, while they still don't have privileged access through the evaluate or wrapped functions. This should be explored as a follow up.

[Jamesernator]

I believe with a fair use case, the Realms API could explore a special method to access buffers

For ArrayBuffer it's more on an optimization thing, but yes it doesn't strictly need priveleged access as you can just proxy it through a membrane.

SharedArrayBuffer is different though, my concern is primarily about the fact this object is effectively rendered useless within a child realm. As an example, if we create a Realm and want to create a fake Worker to expose to the child realm, we can actually faithfully implement basically all of structured clone in JS. The one exception is SharedArrayBuffer, which relies on having a real host object.

In code, this could look imagine creating a fake worker along these lines, however without a way to get SharedArrayBuffer out of the child realm, there's no way to pass it to the real worker:

const childRealm = new Realm();

const createWorker = childRealm.evaluate(`
  ((createWorkerChannel) => {
    globalThis.Event = class Event {
      // ...
    }
    
    globalThis.EventTarget = class EventTarget {
      // ...
    }
    
    function structuredSerializeForTransfer(value) {
      if (typeof value === "number") {
        return JSON.stringify({ kind: "number", value });
      }
      // Other cases ...
      else if (value instanceof ArrayBuffer) {
        const bytes = [...new Uint8Array(value)];
        // Resizable array buffer proposal adds transfer method, so this
        // can be done faithfully
        value.transfer(0);
        return JSON.stringify({ kind: "ArrayBuffer", bytes });
      } else if (value instanceof SharedArrayBuffer) {
        // This cannot be emulated, if we can't pass it out of the Realm, we can
        // do absolutely nothing faithfully here
      }
    }
    
    function structuredDeserialize() {
      // ... Undo structuredSerializeForTransfer, obviously this could be in a module
      // and imported both into this Realm and into the parent rather than duplicated
    }
  
    globalThis.Worker = class Worker extends EventTarget {
      #postMessage;
      constructor(src) {
        this.#postMessage = createWorkerChannel(String(src), (message) => {
          this.dispatchEvent(new Event("message", {
            data: structuredDeserializeSomehow(message),
          }));
        });
      }
      
      postMessage(message, transferables=[]) {
        const message = structuredSerializeForTransfer(message);
        this.#postMessage(message);
      }
      
      terminate() {
        this.#postMessage(null);
      }
    }
  });
`);

function structuredDeserialize() {
  // ... Undo structuredSerializeForTransfer from in Realm
}

createWorker((url, onMessage) => {
  const worker = new Worker(url);
  worker.onmessage = ({ data }) => {
    const result = structuredSerialize(data);
    onMessage(result);
  }
  return (data) => {
    if (data === null) {
      worker.terminate();
      return;
    }
    worker.postMessage(structuredDeserialize(data));
  }
});

[leobalter]

The champion group discussed this and we mentioned it at the TC39 plenaries.

I acknowledge the current API cannot yet pass through buffers but that's something we are looking forward to see as a follow up proposal, even better if anyone is interested in championing it! I believe a new method might do the trick.

[leobalter]

@Jamesernator PTAL at #336? We might end up championing that as a new proposal.

[Jamesernator]

@Jamesernator PTAL at #336? We might end up championing that as a new proposal.

Yeah generally looks good to me for the purpose.