OSM localities joins NE and prefers NE population to calculate population_rank

[peitili]

Background

#2020

When zooming across the NE <> OSM boundary around zoom 8, townspots shift position, population_rank values change, and min_zoom values change.

For country labels in the same places theme we do a join on WikidataID and apply the NE min_zoom to the OSM features.

We should do the same type of harvesting for localities. Namely:

OSM localities should get their population_rank and min_zoom values from NE.
The population value should not be adjusted, as that's for the incorporated area, while the NE population_rank is for the metro area and is more useful for label grading in the stylesheet.

Changes

• Add export functions to export __ne_max_zoom, __ne_min_zoom, __ne_pop_min and __ne_pop_max to the json object
• Move the estimate population value that introduced in Estimate population for null pop OSM places #1993 to a post_process function as the final fall-back values for population
• Use the population values from NE to calculate population_rank field
• Fix a bug that {"population": None} in the json object doesn't end up getting the default estimate population value.

Existing Test Changes

1 Why there is a change necessary for
integration-test.988-add-population-to-collision-rank.PopulationRankTest.test_0 ?

In fact, this test breaks on master too, and this PR fixes it.

2 Why there is a change necessary for
integration-test.988-add-population-to-collision-rank.PopulationRankTest.test_collision_rank_decreasing ?

When the data has tag population: None it falls into following filter line in places.yaml (surprisingly?!)

vector-datasource/yaml/places.yaml

Line 144 in 80799e7

- filter: {name: true, population: true, place: [city, town]}

and thus it won't have the hardcoded estimate population value 10000 for kind_detail: city, it will have population value be 0.

Because we moved the estimation population backfill logic from the yaml file to the transform function, and the transform function is executed at a later stage than the yaml file logic, now when tag population: None happens, it gets a chance to take the hardcoded estimate population value 10000. And thus the collision_rank value changes and as a result, original test breaks. So one way to make this test pass again and more reasonable is to not start from tag population: None but from population: 0.

• Update tests
• Update docs

[peitili]

I am going to do a mini build on California to generate some rendered tiles to verify the change.

[nvkelso]

LGTM but let's see it in a mini build

The CI tests failing are the olds ones, not the new ones.

[nvkelso]

LGTM included Q&A in a mini build.