Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add .pmtiles file support #517

Merged
merged 114 commits into from
Jan 8, 2023
Merged

Add .pmtiles file support #517

merged 114 commits into from
Jan 8, 2023

Conversation

nyurik
Copy link
Member

@nyurik nyurik commented Dec 13, 2022

Merge after #548

Adds a new .pmtiles backend. Supports all formats like png, vector, etc.

From CLI, can be as easy as adding a path to a directory that contains a .pmtiles file:

# All *.pmtiles files in this dir will be published.
# The filename will be used as the source ID
martin ./tests/fixtures

From configuration file, the path can be specified in a number of ways:

pmtiles:
  paths:
    # scan this whole dir, matching all *.pmtiles files
    - /dir-path
    # specific pmtiles file will be published as pmtiles2 source
    - /path/to/pmtiles2.pmtiles
  sources:
      # named source matching source name to a single file
      pm-src1: /tmp/pmtiles.pmtiles
      # named source, where the filename is explicitly set. This way we will be able to add more options later
      pm-src2:
        path: /tmp/pmtiles.pmtiles

Fixes #508

nyurik added 30 commits December 6, 2022 21:41
* All tests and internal code now uses ST_TileEnvelope function
* Remove `tile_bbox`
* Rename test function sources for clarity - this will be needed in a subsequent PR to add other function tests
Can handle the `getmvt(z,x,y) -> [bytea,md5]` case and several others.
* All table sources now use the same system as functions
* Proper table name escaping
* Prepared statements might speed things up, but this is not certain because we prepare on each get request. At least now we can optimize in one place.
BREAKING:
* srid is now the same type as PG -- i32
* renamed config vals `table_sources` and `function_sources` into `tables` and `functions`

Fixes:
* predictable order of instantiation
* bounding boxes computed in parallel for all tables (when not configured)
* split up discovery and query creation - this way user overrides happen before the final query is generated
* more proper name escaping
* lots of test improvements
* Fix id column issue
* Detect if postgis is newer than 3.1 and use st_tileenvolope margin param
* Use `PathBuf` instead of `String` where dealing with files
* Parse keep_alive as u64
* More config tests to crash if martin output contains warnings or errors
* Use `PathBuf` instead of `String` where dealing with files
* Parse keep_alive as u64
* More config tests to crash if martin output contains warnings or errors
* Use `PathBuf` instead of `String` where dealing with files
* Parse keep_alive as u64
* More config tests to crash if martin output contains warnings or errors
* Use `PathBuf` instead of `String` where dealing with files
* Parse keep_alive as u64
* More config tests to crash if martin output contains warnings or errors
* Since this is a library, all errors should have a strongly typed enum.
* table bounds computing function was not escaping identifiers
* table bounds computation was also silently ignoring all errors
@nyurik nyurik marked this pull request as ready for review January 6, 2023 02:10
@nyurik nyurik requested a review from stepankuzmin January 6, 2023 02:11
nyurik added 5 commits January 5, 2023 22:26
Splitting from maplibre#517
* introduce a new Connections object to track all positional strings passed as the CLI arguments
* make xyz use better types - u8 for zoom, u32 for x&y.  Postgres casts those to INT2 and INT8
* minor bug in pre-push git hook
@nyurik nyurik merged commit ba65e34 into maplibre:main Jan 8, 2023
@nyurik nyurik deleted the pmtiles branch January 8, 2023 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for .pmtiles file format
3 participants