WMS url rewrite when rapidmapping from wms-import

[AFoletti]

With the rising rapidmapping use, the danger that permalinks using the non-cached version of the layers is greater.

Today, the cached WMS solution, must be explicitly activated by adapting the WMS url (wms.geo.admin.ch --> wms{s}.geo.admin.ch See #4256). Without this manual workaround, any rapidmapping event distributed to a wide public via map.geo.admin.ch endangers the WMS infra

Solution as discussed with @davidoesch
Force an URL rewrite from wms.geo to wms{s}.geo when a rapidmapping layer is active.

Up for discussion!

[gjn]

Yes, this is quite important. We could even imagine applying wms-c for all wms layers in map.geo. But for sure start with the rapid mapping layers. Pascal could try this one when away from the testviewer (which remains highest prio)

…

On Friday, January 18, 2019, Ambrogio Foletti wrote: With the rising rapidmapping use, the danger that permalinks using the non-cached version of the layers is greater. Today, the cached WMS solution, must be explicitly activated by adapting the WMS url (wms.geo.admin.ch --> wms**{s}**.geo.admin.ch See #4256). Without this manual workaround, any rapidmapping event distributed to a wide public via map.geo.admin.ch endangers the WMS infra Solution as discussed with @davidoesch Force an URL rewrite from wms.geo to wms**{s}**.geo when a rapidmapping layer is active. Up for discussion! -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #469

[AFoletti]

I thought about making it a standard too, but then the layers with a fast update cycle came to mind. We cannot cache tiles for, say, hydroweb WMS.
Of course integration of the hydroweb layers via WMS import in not an important use case but anyways...

So, any plan about that? Q1? Q3? Q1 2032?

[davidoesch]

Q1

[procrastinatio]

Not sure to understand correctly.
rabidmapping is available on wms.geo.admin.ch (not cache) and wms[s].geo.admin.ch (cloudfront cache ?). You want map.geo.admin.chto rewrite fron non cached to cached ?

Sorry, but if rabidmappingmay break something somewhere, it should not be there in the first place.

And why don't you use WMTS and/or the new dynamic layersConfig ?

Layer 'rapidmapping', Single domain, single tile

Layer 'rapidmapping2', multiple domains, tiled WMS tile

See the config (first two layers)
https://mf-geoadmin3.int.bgdi.ch/rapidmapping-configs/de/layersConfig.json

Layer configuration should be in a configuration file, not in the application's black magic.

[AFoletti]

Thanks @procrastinatio for your intervention. I try to answer point by point.

Not sure to understand correctly.
rabidmapping is available on wms.geo.admin.ch (not cache) and wms[s].geo.admin.ch (cloudfront cache ?). You want map.geo.admin.chto rewrite fron non cached to cached ?

Yes, that was the idea. It seemed a good one at the time. Is it no more? Well, great that we have something better!

Sorry, but if rabidmappingmay break something somewhere, it should not be there in the first place.

...or the system could be adapted to accommodate the new requirement (here multiple heavy, potentially very wide usage WMS layers).
However, our WMS infra does not autoscale, and that's indeed not the topic here. It was just to say that I agree in principle but disagree in the present, concrete case

And why don't you use WMTS and/or the new dynamic layersConfig ?

If you look at the ticket opening date, the new dynamic layersConfig was still nonexistant (nor really planned. Or I at least was unaware of it).
What you propose is great. But please remember that it was not an option back then, so no need to ask the question as if it was obvious that we should have gone down that way from the beginning

Layer 'rapidmapping', Single domain, single tile

Layer 'rapidmapping2', multiple domains, tiled WMS tile

See the config (first two layers)
https://mf-geoadmin3.int.bgdi.ch/rapidmapping-configs/de/layersConfig.json

Again, that's great. But the rapidmapping data layers are not chsdi layers (they are not in the BOD). How does it work in this case? How did you build the config for the layers?

Layer configuration should be in a configuration file, not in the application's black magic.

Here, I am 100% with you.
At the time I did not though it was possible and I am still not sure how it will work today. But I admit that I missed this dynamic layerConfig recent developments.
I will come to you tomorrow so that you can explain it to me in detail

Again, thanks for your input. The less bricolage we have the better and I am the first one that needs an eye-opener more often than not.
So, keep it coming!

[AFoletti]

Had an interesting talk with @procrastinatio . I try to summarize the points here, since this was always meant as a discussion ticket. I encourage @gjn , @ltclm, @danduk82 and every other interested party to throw their 2 cents in.

1. the dynamic configuration is truly interesting. It will not replace today's (still necessary) dynamic WMS configuration but opens up interesting avenues (a dynamic rapidmapping topic, for example). Those are however, in my eyes, all just NTH for the rapidmapping system
2. the original problem as stated in the ticket still stands: how to avoid that a rapidmapping layer gets integrated into map.geo.admin.ch via simple "wms import" (and thus going singletile and always tapping the WMS instead of the wms{s} solution which forces tiled WMS with cache) endangers our WMS? The original idea was to force map.geo.admin.ch to always use tiled-WMS for rapidmapping layers, and to this problem I do not see a solution in @procrastinatio 's proposal
3. it would maybe be safe/clever to isolate the rapidmapping WMS service. Since we can control what mapgeoadmin does with it, but not what other web/fat clients do. Those will always use the uncached WMS. Isolating it would allow for wms-bgdi to live happily even if rapidmapping falls

[danduk82]

actually, wms.geo.admin.ch is pointing to cloudfront too, if we use wms{s}.geo.admin.ch is for parallel access to tiles using multi-domain parallelization.

But for cloudfront cache to be efficient it must use WMS-C, yes.

Note that this does not play against using WMS-C also for layers which need fast update, as we use standard cache behavior in cloudfront, whith

min-ttl = 0s
default-ttl = 3600s
max-ttl = 86400s

which means that if a layer needs a smaller TTL than one hour, we can simply set it on backend side. This will provide the benefits of cloudfront "smoother" if there is a peak in requests (WMS-C, of course) and still allow for other resources to have a greater retention time.

[ltclm]

per layer ttl in wms-bgdi backend can be configured in apache config:
https://github.com/geoadmin/wms-bgdi/blob/master/wms-bgdi.conf#L82

[AFoletti]

Am I understanding it right? It would be possible/clever/manageable to make geoadmin to always use tiled WMS when the request goes to wms.geo.admin.ch and the WMS layer is integrated via the import tool?

[danduk82]

Actually, this is what I requested one year ago, and yes it is possible. Could someone please find out the list of WMS layers which are not WMS-C? Maybe we find some cases where there are problems, but I don't think so.
ping @ltclm @rebert

[rebert]

@danduk82
List of non WMS-C:

SELECT * FROM re3.layers_js WHERE layertype like '%wms%' and singletile order by fk_id_dataset;

[AFoletti]

@danduk82
List of non WMS-C:

SELECT * FROM re3.layers_js WHERE layertype like '%wms%' and singletile order by fk_id_dataset;

Yes, that's easy. But those are chsdi layers.
Here the main object are those wms-bgdi layers integrated into mapgeoadmin via wms-import tool (thus bypassing the chsdi configuration)

[rebert]

hmmm