PG TileJSON changes, add vector_layers

[nyurik]

• make tilejson's name be the same as the ID of the source (even if aliased)
• /catalog will always show ID, but now it will hide the name if it is the same as the id
• make description be the longer version, e.g. public.table.column format - not guaranteed to be stable
• make vector_layers have the fields auto-discovered in the PG table
• preserve the order of the serialized json fields

Fixes #583

[codecov-commenter]

Codecov Report

Base: 59.13% // Head: 59.19% // Increases project coverage by +0.05% 🎉

Coverage data is based on head (891848c) compared to base (6a55956).
Patch coverage: 76.47% of modified lines in pull request are covered.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #584      +/-   ##
==========================================
+ Coverage   59.13%   59.19%   +0.05%     
==========================================
  Files          30       30              
  Lines        2736     2747      +11     
==========================================
+ Hits         1618     1626       +8     
- Misses       1118     1121       +3     

Impacted Files	Coverage Δ
src/pg/config.rs	86.73% <ø> (ø)
src/pg/configurator.rs	42.40% <0.00%> (-0.52%)	⬇️
src/pg/config_function.rs	96.77% <100.00%> (ø)
src/pg/config_table.rs	97.43% <100.00%> (+0.66%)	⬆️
src/srv/server.rs	78.66% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[korywka]

Thank you for such fast implementation!

My 5 cents:

I do not know the reason of id, name and description fields. What they shoud tell to me, as a developer.

The only value I need is what to put after / in URL to get a tilejson. It can be table name, function name or others entities from config.yaml.

First, it is not clear if martin makes them automatically unique. For example, if I have table points and function points.
I think that martin doesn't garantee source ids (URL slugs) to be unique. If so, I do not understand why we need schema and column in string templates.

If this is for developer to know how the data is retrieved (e.g. developer have points table and points function, so martin needs to tell from where exaclty this data is fetched), we can create user-friendly text, like:
This source contains geometries from public schema, points table, geometry column. (yep, my english is not perfect, description could be better). But now public.points.geometry looks like some system info, that we need to put somewhere in map SDK :)

So, my minimal /catalog is something like:

[
   {
      "id":"points", // <-- THIS is URL slug
      "content_type":"application/x-protobuf"
   },
   {
      "id":"polygons", // <-- THIS is URL slug
      "content_type":"application/x-protobuf"
   }
]

name and description are unnecessary for me, cause I do not scan all the DB and directly set sources in config.yaml, but yes, I think that name or description would be nice for those who use auto scan feature.

My favorite tilejson like:

{
   "tilejson":"3.0.0",
   "tiles":[
      "https://example.com/source_points/{z}/{x}/{y}"
   ],
   "bounds":[],
   "vector_layers": [] // <-- HERE we know source-layer value for map SDK
   "maxzoom":30,
   "minzoom":14
}

We can add to tilejson id for consistency with /catalog, this would be the same slug from URL.

Source layer id inside vector_layers we can make the same as table name, to easy find out it.

Source layer id should be unique as far as I know inside source only, and the table source can't have multiple source layers (isn't it?)

tilejson can contain some description field the same as inside /catalog to describe how this data is fetched.

In conclusion: I would remove name, make description user-friendly and not system-like, and use only ids: ids are the same as slug URL, vector_layers ids are for source-layer in addLayer method.

PS
I haven't used martin for almost a year now, I missed everything that was updated in the last year. So please take this message only as the opinion of a person who updated martin a year later, I never claim to be an expert 🤝

BTW congrats to maplibre on hosting such great tool, and congrats to martin on the new attention it deserved 🙌

[nyurik]

i removed the dup name if identical to the id. Martin guarantees unique IDs, see

martin/src/source.rs

Lines 71 to 75 in 6a55956

	/// If source name already exists in the self.names structure,
	/// try appending it with ".1", ".2", etc. until the name is unique.
	/// Only alphanumeric characters plus dashes/dots/underscores are allowed.
	#[must_use]
	pub fn resolve(&self, name: &str, unique_name: String) -> String {

[korywka]

If you remove name, should this part be updated with {source_id}?

[nyurik]

definitly, thx!

[nyurik]

fixed

[korywka]

And what is the reason of name? Looks like README is missing this clarification. id and description should be enough, isn't it?

[nyurik]

TileJSON does not define id for the top level, only for the layer. See https://github.com/mapbox/tilejson-spec/tree/master/3.0.0

[korywka]

Oooh. I see. Thanks. Maybe id is not the best naming, and it should be name both for /catalog and /souce_name in the future?
At least now I understand why we need both.

[korywka]

For example, id and name would become name and description. For now it is confusing a bit and you really need to know tilejson spec (as you pointed out to me) to understand this :)

[nyurik]

well, if it is called name, just like thought before - people don't know if it is unique. Name on the other hand could come from tilejson stored in the tile storage itself, e.g. inside the .mbtiles file's metadata.

I'm not sure where to add info about source ID uniqueness to the readme - suggestions welcome.

[korywka]

It would be nice to describe all possible keys from catalog item json here: https://github.com/maplibre/martin#source-list
What is the id (+unique), name and description. Why sometimes name is missing (after this PR).

If I understood you correctly and catalogItem.id is the same as tilejson.name - this is also helpful information (we also can say the reason - tilejson can't have id).

[nyurik]

name and id are not the same. TileJSON is a way to describe a tile source. Name comes from the source. ID on the other hand is the slug used in the URL - its a concept that Martin has. ID is configured by the Martin server's configuration. Multiple IDs can exist for the same source. The name could be the same as the source ID, but could be different - e.g. see one of the mbtiles tests:

 {
    "id": "geography-class-png-no-bounds",
    "content_type": "image/png",
    "name": "Geography Class",
    "description": "One of the example maps that comes with TileMill - a bright & colorful world map that blends retro and high-tech with its folded paper texture and interactive flag tooltips. "
  },

[korywka]

Thank you for clarification. Would be nice to have this information in README 🙏
I understand that martin is not for beginners, but I think it's simple and obvious only for martin developer.
Maybe it's just me getting confused 😇
We can say that name is this one field: https://github.com/mapbox/tilejson-spec/tree/master/3.0.0#314-name
Description is this one: https://github.com/mapbox/tilejson-spec/tree/master/3.0.0#38-description
And id is a unique value for URL slug (autogenerated?).
And name can be missing if it is the same as id (do not know the reason of this decision, but it is worth clarifying)

[nyurik]

PRs for the readme are more than welcome :)))))) or just post here what text you want to see where, and i can add it to the pr and clean up the language

[korywka]

Do not know if you want to have russian/ukranian-styled english in README 🥲
But one of these days I may try to do it :)

[nyurik]

improving bad style english is far easier than to write new content from scratch ;)

[korywka]

Do not know is it true, maybe these parts should also use source_id:

And this one?

I think the one too:

It is table names for auto scan, but does not always table names if you use config or tables are duplicated. IMHO it is better to say source_id, which is by default table name, but can be changed in config or incrimented while duplicate (this part is present in readme)

As far as I know martin will take table_source_id key from config as a source_id and NOT table (which is really table name). Looks like README is more auto scan oriented.

I can try to fix it too if this is true.

[nyurik]

yes, i think those should be modified - could you maybe start a new PR for that? You don't even need to clone it, can use the github's built-in editor for the readme. In some cases {source} and {source1} might be OK I think, but in some cases you may say "/{sourceID}/{z}/{x}/{y}, where sourceID is the table name by default, unless modified in the configuration or conflicts with another source with the same name."

Thanks!

[stepankuzmin]

What's that filter for?

[nyurik]

hides the name from the output if it is identical to the ID

[stepankuzmin]

LGTM 👍