Request performance improvements

[lilleyse]

Any request prioritization related PRs will be going into this one.

Task list:

• Request priority data structure Change request queue from a heap to a sorted array #6240
  • Is a sorted array performant enough if the request queue gets larger? Move to a double-sided priority queue instead?
  • Queue up requests and then sort? Change request queue from a heap to a sorted array #6240 (comment)
• Implement Request pool to avoid allocating many Request objects.
• Fix 3D Tiles starving terrain/imagery: RequestScheduler starves imagery requests for 3D Tiles #5954
• 3D Tiles improvements 3D Tiles - Request scheduling improvements #5509
  • Time slice processing - which also means sorting received tiles by priority
  • Cull received tiles that are now out of view
  • Cancel request when tile goes out of view
• Tweaks for better http2 performance. Some notes in Tile Request Prioritization #5317
• Different prioritization modes
  • SSE
  • Close to center of screen
  • Distance (current approach)
  • How can we mix and match different prioritization modes. Additive tiles use distance, replacement tiles use SSE, terrain uses distance but should probably use SSE. Is there any way to normalize all values to a 0-1 range?
• Stagger renders to allow more request throughput: https://groups.google.com/forum/#!topic/cesium-dev/WQ4mgLcoBqI
• skip lods for additive refinement
  • Parent tile keeps flag indicating whether descendants have renderable content
• Debug visualizations to see load order.
  • Colorize by load order
  • Secondary camera see loaded tiles from a better vantage points
• Evaluate performance for smaller or larger tile sizes
• For terrain
  • Heuristic to take into account dot(view-direction, tile surface normal) so that the more the viewer is looking at the tile, the more important it is?
  • Sorting terrain requests by distance is probably bad and messes with terrain's internal priority handling.

[cesium-concierge]

@lilleyse, thanks for the pull request! Maintainers, we have a signed CLA from @lilleyse, so you can review this at any time.

---

I am a bot who helps you make Cesium awesome! Contributions to my configuration are welcome.

🌍 🌎 🌏

[mramato]

Will any of these improvements take HTTP/2 vs HTTP/1.1 throughput into account? For many of these optimizations, I'm sure it doesn't matter but I've seen incredible performance improvements with HTTP/2 for certain views if I throttling is disabled completely. I guess an optimal scheduling system won't actually care which protocol is being used.

[lilleyse]

Yes, I have a note for http2:

Tweaks for better http2 performance. Some notes in #5317

[mramato]

Shame on me, sorry for the noise I totally missed that.

[cesium-concierge]

Congratulations on closing the issue! I found these Cesium forum links in the comments above:

https://groups.google.com/forum/#!topic/cesium-dev/WQ4mgLcoBqI

If this issue affects any of these threads, please post a comment like the following:

The issue at #6243 has just been closed and may resolve your issue. Look for the change in the next stable release of Cesium or get it now in the master branch on GitHub https://github.com/AnalyticalGraphicsInc/cesium.

---

I am a bot who helps you make Cesium awesome! Contributions to my configuration are welcome.

🌍 🌎 🌏

[cesium-concierge]

Thanks again for the pull request!

Looks like this pull request hasn't been updated in 30 days since I last commented.

To keep things tidy should this be closed? Perhaps keep the branch and submit an issue?

I am a bot who helps you make Cesium awesome! Contributions to my configuration are welcome.

🌍 🌎 🌏

[kring]

Different prioritization modes

For the terrain improvements I've been working on, I've experimented with this metric for terrain tile prioritization:

        var cameraPosition = frameState.camera.positionWC;
        var cameraDirection = frameState.camera.directionWC;
        var tileDirection = Cartesian3.normalize(Cartesian3.subtract(obb.center, cameraPosition, tileDirectionScratch), tileDirectionScratch);
        return (1.0 - Cartesian3.dot(tileDirection, cameraDirection)) * tile._distance;

And have been pretty happy with it so far, though I haven't done a particularly principled analysis of it.

FYI, though, I've also disabled terrain tile prioritization with RequestScheduler cause it just doesn't do anything like what I want right now (for one thing: terrain has a low/medium/high priority load queue, which the Scheduler ignores and just looks at tile distance) and there's a bunch of overhead (for one thing: per-tile priorityFunction).

[lilleyse]

Hey @kring I noted this problem above:

Sorting terrain requests by distance is probably bad and messes with terrain's internal priority handling.

This is probably one of the bigger problems with the RequestScheduler right now. Originally we wanted a single metric to compare all requests, and distance was the easiest choice, but I don't think it's a good idea anymore. In #6390 additive tiles still use distance, but replacement tiles use SSE, and terrain tiles have their own priority handling, so distance just doesn't work across the board.

I envision RequestScheduler being a lot more trimmed down. Maybe a limit on how many requests can be active for each type, but none of the sorting that it does now. #6390 updated the 3D Tiles code to issue requests in order of priority, rather than arbitrarily like in master, so that's one step closer...

You priority function also looks a lot like @pjcozzi's suggestion in the task list above:

Heuristic to take into account dot(view-direction, tile surface normal) so that the more the viewer is looking at the tile, the more important it is?

[kring]

Maybe a limit on how many requests can be active for each type, but none of the sorting that it does now.

Makes sense to me. You might be interested in taking a look at my slightly hacky changes to Request and RequestScheduler in the bvh branch: master...bvh

The idea is to make throttleByServer independent of throttle. So when throttleByServer is true but throttle is false, you get the usual server-level request limiting (and it returns undefined when you exceed it), but it never queues or sorts requests.

You priority function also looks a lot like @pjcozzi's suggestion in the task list above:

Not quite, my metric is dot(direction-from-camera-to-center-of-tile, camera-view-direction), so tiles near the center of the screen are given higher priority (when distances are similar). Nothing to do with tile surface normal. Really it's not that different from sorting by distance, except that it tends to deprioritize tiles that are nearly off screen and may not even be visible, but we think they are because their bounding volume just barely intersects the view frustum.

[pjcozzi]

@ggetz is it worth considering some of these?

[lilleyse]

Not quite, my metric is dot(direction-from-camera-to-center-of-tile, camera-view-direction), so tiles near the center of the screen are given higher priority (when distances are similar).

Makes sense - I read the code snippet too quickly before. We've been planning on adding something similar to the 3D Tiles traversal.

The request scheduler changes sound good too... back to basics.

[lilleyse]

I was originally hoping to finish more of these and consolidate them into a single PR (this), but I think it's better now to close this and submit fixes individually to master. I moved 3D Tiles relevant items to #5509.

[cesium-concierge]

Congratulations on closing the issue! I found these Cesium forum links in the comments above:

https://groups.google.com/forum/#!topic/cesium-dev/WQ4mgLcoBqI

If this issue affects any of these threads, please post a comment like the following:

The issue at #6243 has just been closed and may resolve your issue. Look for the change in the next stable release of Cesium or get it now in the master branch on GitHub https://github.com/AnalyticalGraphicsInc/cesium.

---

I am a bot who helps you make Cesium awesome! Contributions to my configuration are welcome.

🌍 🌎 🌏