Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Postpone disposal when there is a pending read #394

Closed
dsmilkov opened this issue Jun 6, 2018 · 0 comments
Closed

Postpone disposal when there is a pending read #394

dsmilkov opened this issue Jun 6, 2018 · 0 comments
Assignees
@dsmilkov
Labels
comp:core type:bug Something isn't working

Comments

@dsmilkov
Copy link
Contributor

dsmilkov commented Jun 6, 2018

Since we don't postpone disposal when there is a pending read, the following happens:

tf.tidy(() => {
  const tensor = tf.scalar(1);
  tensor.data().then(vals => {
    // vals is null here.
  });
});
@dsmilkov dsmilkov added type:bug Something isn't working comp:core labels Jun 6, 2018
@dsmilkov dsmilkov self-assigned this Jun 6, 2018
@dsmilkov dsmilkov mentioned this issue Jun 6, 2018
Merged
dsmilkov added a commit to tensorflow/tfjs-core that referenced this issue Jun 6, 2018
@dsmilkov
WebGL Backend: Centralize async reading of data, postpone disposal if…
78850e1 
… a pending read (#859)

BUG
FEATURE
PERF

All the work involves only the WebGL backend.

- Centralize async reading of data in a single setTimeout loop
- Postpone disposal if there is a pending read
- Use LRU cache to page textures out of the GPU memory and into CPU (Avoids crashing browser)
- Some optimizations:
  - Cache the `ext.GPU_DISJOINT_EXT` parameter since reading it is expensive
  - Do binary search for queryIsDone since checking is expensive
  - Reuse download buffers for readPixels to minimize js memory trashing

Did a stress test using demos/mnist_eager where I made tensor.dispose() a no-op. There is significant slowdown (~10x) due to constant downloads and js GC trashing memory, but the browser doesn't crash. The optimizations I did were motivated by the profiling from this stress test.

Fixes tensorflow/tfjs#394
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dsmilkov dsmilkov

Labels
comp:core type:bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@dsmilkov