Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discuss when APIs that take a buffer to write into are considered A-OK #231

Open
annevk opened this issue Aug 3, 2020 · 4 comments
Open
Assignees

Comments

@annevk
Copy link
Member

annevk commented Aug 3, 2020

For performance-sensitive operations it can sometimes be important that the web developer is in charge of allocating memory. APIs that take a buffer can be designed for such cases (also known as "outparams"). https://encoding.spec.whatwg.org/#dom-textencoder-encodeinto has an example. I believe Web Audio has a similar API.

It would be good to discuss this in the document as an acceptable solution for performance-sensitive APIs. It might also be good to discuss the constraints. At least when discussing this for Encoding we decided that throwing an exception after some computation had already happened was not acceptable (we used a return value to encode that information instead).

cc @padenot

@padenot
Copy link

padenot commented Aug 3, 2020

An assorted list of problems caused by not using this pattern:

  • Other patterns force object allocation, and prevent having zero-gc path. Real-time workloads (wether soft or hard, say, requestAnimationFrame vs. AudioWorkletProcessor.process) are especially susceptible to this problem. GC in JavaScript engines are very fast these days, but they are still clearly unsuitable for real-time workloads
  • Other patterns force memory allocations (say, by returning a typed array), or forces implementations to implement needlessly complex memory pooling or retainment policy (AudioWorkletProcess.process parameters is a good example of this, not retaining the memory behind the parameters prevented the API from working as it should, details in Improve description of parameters for process() calls on client objects extending AudioWorkletProcessor WebAudio/web-audio-api#1933)
  • Other patterns force copies when used with WASM or other code that manages its own memory, this in turns prevents writing code that takes advantage of memory locality for speedups, and therefore impedes composability with high-performance code

An example of a well designed API in this context would be AnalyzerNode.getFloatFrequencyData: this is expected to be called in requestAnimationFrame, with a buffer of the same size each time, it's natural to pass a buffer to get the values at a particular time.

This should go further than allowing passing in memory. It should specifically require that functions that use Typed Arrays of any sort must use this pattern. This allows maximal performances, for when it matters, and API designer at the design stage cannot assume a particular context of use. It also steers authors in the way of caring about performances, and potentially teaches them about the issues described above.

@Manishearth
Copy link

There's some discussion of this for WebXR in immersive-web/webxr-hand-input#37 . Basically, to avoid recreating 3-5 objects per joint (for 50 joints), we want to introduce an API that has outparams, either of the form:

// setup
let set = new XRJointSet(inputSource.hand, ...);
// each frame
frame.populatePoses(set, ...);
for (joint in joints) {
   let jointPosition = set[joint].position; // or `set.position[joint]`
   let jointOrientation = set[joint].orientation; // or `set.position[joint]`
}

// OR (this one is more friendly to sending the data directly to the GPU)
// setup
let positions = new Float32Array();
let orientations = new Float32Array();

// each frame
frame.getPoses(inputSource.hand.joints, positions, orientations);
for (joint in joints) {
   let jointPosition = positions[joint * 4, joint* 4 + 4];
   let jointOrientation = orientations[joint * 4, joint* 4 + 4];
}

it would be nice to know if either of these is acceptable, and if the TAG has a preference between the two.

@annevk
Copy link
Member Author

annevk commented Aug 4, 2020

(WebGL uses this too, didn't look where.)

@toji
Copy link

toji commented Aug 4, 2020

readPixels() is an example of a WebGL API that fills a application-supplied array with data. It should be noted that since a design goal for WebGL was to be ~1:1 mapping of the C API there are naturally quite a few C-isms that came along with it. For that same reason, though, nobody points at WebGL as a shining example of API ergonomics. 😉

That said, I personally would very much appreciate having some platform-approved patterns available for APIs that want to allow developers to minimize object churn.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants