ShapeFile no longer accessible

[HekMe]

Hello,
this shapefile http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip is no longer accessible, which is needed by this script - https://github.com/gravitystorm/openstreetmap-carto/blob/v5.1.0/scripts/get-shapefiles.py
At this point of your docs https://github.com/Magellium/osmtilemaker#generate-the-tiles running this scripts returns timeout errors, and doesn't continue with downloading other shapefiles(just first of them).
I overcame it by copying those missing shapefiles from other of my projects to destination folder.
I've also found this https://github.com/nvkelso/natural-earth-vector/tree/master/110m_cultural, but it contains a lot more than those basic ne_110m_admin_0_boundary_lines_land.

[audour]

Thanks for pointing the issue.

It seems related to these closed issues : nvkelso/natural-earth-vector#528 and gravitystorm/openstreetmap-carto#4249
This replacement link seems to work: http://naturalearth.s3.amazonaws.com/110m_cultural/ne_110m_admin_0_boundary_lines_land.zip

Current osmtilemaker source code is using https://github.com/gravitystorm/openstreetmap-carto/releases/tag/v5.1.0 (see #1).
I'm currently upgrading to the last version https://github.com/gravitystorm/openstreetmap-carto/releases/tag/v5.3.1 where get-shapefiles.py has been replaced by get-external-data.py. Whatever in the corresponding configuration file (https://github.com/gravitystorm/openstreetmap-carto/blob/v5.3.1/external-data.yml), the url is now https://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip which is also not working for me ...

So the issue seems more related to https://github.com/nvkelso/natural-earth-vector and https://github.com/gravitystorm/openstreetmap-carto

Here are the logs with the last openstreetmap-carto version:

root@3aae53238a55:/docker_mounted_volumes/working_dir/openstreetmap-carto# $DOCKERPATH_WORKING_DIR/openstreetmap-carto/scripts/get-external-data.py -d $DBPG_DATABASE_NAME -U $DBPG_USER_OSMTILEMAKER_USERNAME
INFO:root:Checking table simplified_water_polygons
CRITICAL:root:ogr2ogr returned 1 with layer simplified_water_polygons
CRITICAL:root:Command line was ogr2ogr -f PostgreSQL -lco GEOMETRY_NAME=way -lco SPATIAL_INDEX=FALSE -lco EXTRACT_SCHEMA_FROM_LAYER_NAME=YES -nln loading.simplified_water_polygons "PG:dbname=osmtilemaker user=osmtilemaker" data/simplified_water_polygons/simplified-water-polygons-split-3857/simplified_water_polygons.shp
CRITICAL:root:Output was

Traceback (most recent call last):
  File "/docker_mounted_volumes/working_dir/openstreetmap-carto/scripts/get-external-data.py", line 275, in main
    ogrcommand, stderr=subprocess.PIPE, universal_newlines=True)
  File "/usr/lib/python3.7/subprocess.py", line 395, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ogr2ogr', '-f', 'PostgreSQL', '-lco', 'GEOMETRY_NAME=way', '-lco', 'SPATIAL_INDEX=FALSE', '-lco', 'EXTRACT_SCHEMA_FROM_LAYER_NAME=YES', '-nln', 'loading.simplified_water_polygons', 'PG:dbname=osmtilemaker user=osmtilemaker', 'data/simplified_water_polygons/simplified-water-polygons-split-3857/simplified_water_polygons.shp']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/docker_mounted_volumes/working_dir/openstreetmap-carto/scripts/get-external-data.py", line 298, in <module>
    main()
  File "/docker_mounted_volumes/working_dir/openstreetmap-carto/scripts/get-external-data.py", line 284, in main
    "ogr2ogr error when loading table {}".format(name))
RuntimeError: ogr2ogr error when loading table simplified_water_polygons

It needs more investigation. Any help welcome :)

[audour]

It turns out to be an upstream issue which is discussed here:

• domain doesn't resolve - naciscdn.org gravitystorm/openstreetmap-carto#4461
• Cannot download naciscdn.org data from naturalearthdata.com nvkelso/natural-earth-vector#581 (comment)

[audour]

I'll close the issue as:

• this PR resolves upstream issue: Fix URL for natural earth vector dataset. gravitystorm/openstreetmap-carto#4466
• the PR has been merged and corresponding changes are available within openstreetmap-carto 5.4.0 release
• I've just published osmtilemaker 1.1 release which includes openstreetmap-carto 5.4.0