-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(vis) layered rendering demo (#22)
* first step to layered rendering: one layer! * bufferPair gets its own file in common/ * technically functional rendering of layers of ome-zarr slices and scatterplot points. * dig up the old ccf average-template data... todo ask scott for better chunk-sizes! * I can load a slide-view slide and the average_template ccf (ask scott for a better one) and show them togeather, although their dimensions suggest that they are radically different sizes * get the dang things to line up by fixing all my goofy voxel math * aaaaaaah what a wild bug * a little less sloppy * fun idea but breaks visibility determination * generalize layers, build up convenience around frames, more formal types to be rendered * a working (but somewhat confusing) generic layer (impl for scatterplots first) * both layer types working * delete half-baked approach * pull out common stuff (target) from generic types to make things less confusing * add a super basic annotation (just lines) renderer, wrap it in a layer thingy * fix a lil slop * minor changes as I prepare to add some sort of UI * working but very strange imgui implementation... lets see if its nice * add some UI - not in love withit * draw after loading * start to merge the two different zarr rendering methods * less hacky way of picking an appropriate resolution, although its still a bit iffy... * thinking about skipping ui and just having json state... * super basic optional transforms for layers. next up a grid of these things * add a volume grid layer, mild cleanup elsewhere * draw data as it arrives - and prepend frames with low-res data to cover the pan case * refactor various types and loaders to make it easier to configure an instance with a list of simple config payloads. separate data types from the code that would render them. updates to demo to use new loaders * lets serve the app with parcel * Get layers app Parcelized * Noah/layers pt2 (#23) * delete ye olde build script * add some react for UI - think about if I like it or it will make my life nice * fix a perf bug in which we consider rendering every slice in a grid regardless of zoom level. some (better?) smoke and mirrors trickery to try to obscure redraw flickering when panning/zooming * working but messy impl of a worker pool that handles decoding zarr chunks for us # Conflicts: # apps/layers/package.json * play around with WW pool size, and fix a bug in which the optional render-working budget parameters were being ignored (for the slice renderer specifically) * move a shocking amount of code in to render slide-view style annotations # Conflicts: # apps/layers/package.json * good enough for now # Conflicts: # apps/layers/src/demo.ts * respect queue params * enable screenshots, with a default output resolution of roughly 85MP # Conflicts: # apps/layers/package.json # apps/layers/src/demo.ts # pnpm-lock.yaml * start thinking about upside down data... * its all upside down now great * minor tidy, plus a rough attempt at a less flickery stand-in algorithm * tools to add layers during demotime # Conflicts: # apps/layers/src/demo.ts * add a versa layer * add scatterplot-slideview real quick # Conflicts: # apps/layers/src/demo.ts * start some cleanup so I can merge this... # Conflicts: # apps/layers/src/demo.ts * Merge branch 'noah/layered-demo' into noah/layered-with-react-whynot # Conflicts: # apps/layers/src/demo.ts * quickly change the underlying cache type for scatterplots for much better perf (gpu buffer not client-buffer) * try out sds components for quick hacky ui fun - delete old ui code * add a bunch of per-layer ui elements * prev/next layer buttons * take a snapshot button * quickly re-enable drawing layers * a bit hacky, but non-flickering drawings are worth it for a demo * change moduleResolution in the apps tsconfig to make the zarr library that we use extensively get resolved correctly. this is an issue on their end: gzuidhof/zarr.js#152 * cleanup some increasingly scary cherrypicks, and finally tidy up those last little demo ts errors. * clean up a bunch of low hanging fruit * fix up the scatterplot (standalone) demo * readme and demo script * a little more * copy in the latest and greatest annotation stuff in * minor cleanups * fix wrongly named example --------- Co-authored-by: Lane Sawyer <[email protected]>
- Loading branch information
1 parent
6fedb9e
commit e10b436
Showing
49 changed files
with
3,713 additions
and
406 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
|
|
||
| export type BufferPair<T> = { | ||
| writeTo: T; | ||
| readFrom: T; | ||
| } | ||
| export function swapBuffers<T>(doubleBuffer: BufferPair<T>) { | ||
| const { readFrom, writeTo } = doubleBuffer; | ||
| return { readFrom: writeTo, writeTo: readFrom }; | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
|
|
||
| // a web-worker which fetches slices of data, decodes them, and returns the result as a flat float32 array, using transferables | ||
| import type { NestedArray, TypedArray } from 'zarr' | ||
| import { getSlice, type ZarrDataset, type ZarrRequest } from "./zarr-data"; | ||
|
|
||
| const ctx = self; | ||
| type ZarrSliceRequest = { | ||
| id: string; | ||
| type: 'ZarrSliceRequest' | ||
| metadata: ZarrDataset | ||
| req: ZarrRequest, | ||
| layerIndex: number | ||
| } | ||
| function isSliceRequest(payload: any): payload is ZarrSliceRequest { | ||
| return typeof payload === 'object' && payload['type'] === 'ZarrSliceRequest'; | ||
| } | ||
| ctx.onmessage = (msg: MessageEvent<unknown>) => { | ||
| const { data } = msg; | ||
| if (isSliceRequest(data)) { | ||
| const { metadata, req, layerIndex, id } = data; | ||
| getSlice(metadata, req, layerIndex).then((result: { | ||
| shape: number[], | ||
| buffer: NestedArray<TypedArray> | ||
| }) => { | ||
| const { shape, buffer } = result; | ||
| const R = new Float32Array(buffer.flatten()); | ||
| ctx.postMessage({ type: 'slice', id, shape, data: R }, { transfer: [R.buffer] }) | ||
| }) | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| import { uniqueId } from "lodash"; | ||
| import type { ZarrDataset, ZarrRequest } from "./zarr-data"; | ||
|
|
||
| type PromisifiedMessage = { | ||
| requestCacheKey: string; | ||
| resolve: (t: Slice) => void; | ||
| reject: (reason: unknown) => void; | ||
| promise?: Promise<Slice> | undefined; | ||
| }; | ||
| type ExpectedResultSlice = { | ||
| type: 'slice', | ||
| id: string; | ||
| } & Slice; | ||
| type Slice = { | ||
| data: Float32Array; | ||
| shape: number[] | ||
| } | ||
| function isExpectedResult(obj: any): obj is ExpectedResultSlice { | ||
| return (typeof obj === 'object' && 'type' in obj && obj.type === 'slice') | ||
| } | ||
| export class SliceWorkerPool { | ||
| private workers: Worker[]; | ||
| private promises: Record<string, PromisifiedMessage>; | ||
| private which: number; | ||
| constructor(size: number) { | ||
| this.workers = new Array(size); | ||
| for (let i = 0; i < size; i++) { | ||
| this.workers[i] = new Worker(new URL('./fetchSlice.worker.ts', import.meta.url), { type: 'module' }); | ||
| this.workers[i].onmessage = (msg) => this.handleResponse(msg) | ||
| } | ||
| this.promises = {}; | ||
| this.which = 0; | ||
| } | ||
|
|
||
| handleResponse(msg: MessageEvent<unknown>) { | ||
| const { data: payload } = msg; | ||
| if (isExpectedResult(payload)) { | ||
| const prom = this.promises[payload.id]; | ||
| if (prom) { | ||
| const { data, shape } = payload | ||
| prom.resolve({ data, shape }); | ||
| delete this.promises[payload.id] | ||
| } | ||
| } | ||
| } | ||
| private roundRobin() { | ||
| this.which = (this.which + 1) % this.workers.length | ||
| } | ||
| requestSlice(dataset: ZarrDataset, req: ZarrRequest, layerIndex: number) { | ||
| const reqId = uniqueId('rq'); | ||
| const cacheKey = JSON.stringify({ url: dataset.url, req, layerIndex }); | ||
| // TODO caching I guess... | ||
| const eventually = new Promise<Slice>((resolve, reject) => { | ||
| this.promises[reqId] = { | ||
| requestCacheKey: cacheKey, | ||
| resolve, | ||
| reject, | ||
| promise: undefined, // ill get added to the map once I am fully defined! | ||
| }; | ||
| this.workers[this.which].postMessage({ id: reqId, type: 'ZarrSliceRequest', metadata: dataset, req, layerIndex }); | ||
| this.roundRobin(); | ||
| }); | ||
| this.promises[reqId].promise = eventually; | ||
| return eventually; | ||
| } | ||
| } | ||
|
|
||
| // a singleton... | ||
| let slicePool: SliceWorkerPool; | ||
| export function getSlicePool() { | ||
| if (!slicePool) { | ||
| slicePool = new SliceWorkerPool(16); | ||
| } | ||
| return slicePool; | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.