writer: Leaf directories: Find best base zoom

[eddy-geek]

... to avoid extra indirection for as many tiles as we can

Previously, we hardcoded a "base_zoom = 7", which is only only optimal for a whole world map.
instead "7" is replaced with min zoom that fits all tiles.

I only tested on one example (Bugianen, 152742 tiles) and got the expected base zoom of 14.

```sh
sqlite3 Bugianen.mbtiles "SELECT  zoom_level, COUNT(*) FROM tiles GROUP BY zoom_level"
12|448
13|1792
14|7168
15|28672
16|114662

PYTHONPATH=$PWD bin/pmtiles-convert Bugianen.mbtiles optim_Bugianen.pmtiles
Num tiles: 152742
Num unique tiles: 151947
Num leaves: 7168

So, this needs more testing.

[eddy-geek]

Cc protomaps/go-pmtiles#6 where this came up

[bdon]

Yeah, let me test this some more. For background, the code as-is is designed for a "full pyramid", even if is not the whole world: what "full" means is that if tile 8, 6, 3 exists in the tileset, then 7, 3, 2 also must exist, and 6, 2, 1 ... etc.

a non-full pyramid means that the map has nothing to show when zooming out past a certain level, or must lock the UI to a minimum zoom, both which aren't a great user experience. Regardless, the PMTiles reference implementations should do something smarter for full + deep pyramids (zoom levels 20+, for example)

[eddy-geek]

• even "full pyramid" of a small area will not be optimal with base_zoom=7
• your point about UX is not valid for overlays
• I now have a second file, 7GB, converted successfully with this PR.

[bdon]

Yeah, that was what I meant - the current very primitive implementation of leafs at 7 is not optimized even for non-global full pyramids. Ideally this can have a reasonable implementation in python with unit tests that I can then port over to the Go and C++ implementations as those don't support writing leaves yet.

Can you give an example of the distinction between overlays and basemaps? My impression is that tilesets will always benefit from including generalizations at lower zooms, at the expense of more work to write + larger archive size.

[eddy-geek]

Sure:

• What's special about overlays is that the basemap is usually used to navigate around, so the overaly can only display information when relevant.

• If you include low zooms anyway, you may make the maps unreadable as in

  • OpenStreetMap + "Hills" Overlay on wtracks
    

  • this swisstopo example
    

• I make a slope overlay eslope, it should only appear at high zoom-levels as it's only useful close to your position/track.

[bdon]

Okay, those are good examples of cases where we don't want a "full" pyramid. When I get to revisiting this I'll make sure there's no optimizations made assuming that the tileset is complete for lower zooms.

[bdon]

Sorry for the delay - merging this now! The JavaScript client needs to be updated to handle leaf levels >7, which I'm hoping to work on this week.

[bdon]

The JavaScript client should handle leaf levels >7, and the python writer has been rewritten to correct edge cases and work more efficiently - please give it a try if you can: (on master, not on PyPI yet) https://github.com/protomaps/PMTiles/blob/master/python/pmtiles/writer.py#L14