Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Harvesting - GeoNode Client] Harvesting of layers data #7945

Closed
meomancer opened this issue Aug 13, 2021 · 0 comments · Fixed by #7947
Closed

[Harvesting - GeoNode Client] Harvesting of layers data #7945

meomancer opened this issue Aug 13, 2021 · 0 comments · Fixed by #7947
Assignees
Milestone

Comments

@meomancer
Copy link
Contributor

Based on geosolutions-it/nexus-geonode#178

meomancer added a commit to meomancer/geonode that referenced this issue Aug 13, 2021
meomancer added a commit to meomancer/geonode that referenced this issue Aug 13, 2021
meomancer added a commit to meomancer/geonode that referenced this issue Aug 13, 2021
@ricardogsilva ricardogsilva linked a pull request Sep 21, 2021 that will close this issue
3 tasks
ricardogsilva pushed a commit to meomancer/geonode that referenced this issue Sep 29, 2021
afabiani added a commit that referenced this issue Sep 29, 2021
* Bump urllib3 from 1.26.2 to 1.26.3 (#6908)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.2 to 1.26.3.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/1.26.3/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.2...1.26.3)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Toni <[email protected]>

* [Fixes #6880] Circle CI upload tests fail irregulary (#6881)

* [Fixes #6880] Circle CI upload tests fail irregulary

* CircleCI test fix: sometimes expires due to upload timeout in the test environment

* - Avoid infinite loop on upload testing

* Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment"

This reverts commit 66139fd.

Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>

* [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915)

* Added malnajdi as contributor

* [Fixes #6910] meaningful filename for document download (#6911)

* get meaningful document filenames on download

* - Strip extension from document title before slugify it (e.g.: image.jpg instead of imagejpg.jpg)

Co-authored-by: afabiani <[email protected]>
Co-authored-by: Alessio Fabiani <[email protected]>

* - CircleCI Upload Tests: trying to reduce more the risk of infinite loop on "wait_for_progress"

* [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION (#6913)

* [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION

* [Fixes #6916] unit test for GEOSERVER_LOCATION

* Bump django-cors-headers from 3.6.0 to 3.7.0 (#6901)

Bumps [django-cors-headers](https://github.com/adamchainz/django-cors-headers) from 3.6.0 to 3.7.0.
- [Release notes](https://github.com/adamchainz/django-cors-headers/releases)
- [Changelog](https://github.com/adamchainz/django-cors-headers/blob/master/HISTORY.rst)
- [Commits](adamchainz/django-cors-headers@3.6.0...3.7.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump amqp from 5.0.3 to 5.0.5 (#6905)

Bumps [amqp](https://github.com/celery/py-amqp) from 5.0.3 to 5.0.5.
- [Release notes](https://github.com/celery/py-amqp/releases)
- [Changelog](https://github.com/celery/py-amqp/blob/master/Changelog)
- [Commits](celery/py-amqp@v5.0.3...v5.0.5)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pip from 21.0 to 21.0.1 (#6900)

Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst)
- [Commits](pypa/pip@21.0...21.0.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump coverage from 5.3.1 to 5.4 (#6903)

Bumps [coverage](https://github.com/nedbat/coveragepy) from 5.3.1 to 5.4.
- [Release notes](https://github.com/nedbat/coveragepy/releases)
- [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst)
- [Commits](nedbat/coveragepy@coverage-5.3.1...coverage-5.4)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pytest from 6.2.1 to 6.2.2 (#6907)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 6.2.1 to 6.2.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/master/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@6.2.1...6.2.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump djangorestframework-gis from 0.16 to 0.17 (#6902)

Bumps [djangorestframework-gis](https://github.com/openwisp/django-rest-framework-gis) from 0.16 to 0.17.
- [Release notes](https://github.com/openwisp/django-rest-framework-gis/releases)
- [Changelog](https://github.com/openwisp/django-rest-framework-gis/blob/master/CHANGES.rst)
- [Commits](openwisp/django-rest-framework-gis@v0.16.0...v0.17.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* - Algin setup.cfg to requirements.txt

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923)

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded

* - Add REST APIs test suite to CircleCI

* [Fixes #6918] Removal of QGIS support (#6919)

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* - Fix LGTM issues

* allow Basic authenticated requests in LOCKDOWN mode

* fix to avoid circular import

* flake8 check fix

* added tests

* [Fixes #6880] Circle CI upload tests fail irregulary (#6881)

* [Fixes #6880] Circle CI upload tests fail irregulary

* CircleCI test fix: sometimes expires due to upload timeout in the test environment

* - Avoid infinite loop on upload testing

* Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment"

This reverts commit 66139fd.

Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>

* [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915)

* Added malnajdi as contributor

* Bump pip from 21.0 to 21.0.1 (#6900)

Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst)
- [Commits](pypa/pip@21.0...21.0.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* - Algin setup.cfg to requirements.txt

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923)

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded

* - Add REST APIs test suite to CircleCI

* [Fixes #6918] Removal of QGIS support (#6919)

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* - Fix LGTM issues

* allow Basic authenticated requests in LOCKDOWN mode

* fix to avoid circular import

* - Align to upstream master branch

* [Fixes #7945] Ingest harvested layer data to geonode

* Refactor celery tasks in order to allow performing harvesting of selected harvestable resources

* Implement copying of remote resources for the GeoNode legacy harvester

* Improve harvesting session and the admin

* Ensure `extension` is present for harvested GeoNode documents

* Add missing default attributes to harvested resources

* fixing tests

* fixing tests

* fix migration files conflict

* Renamed `geonode` harvester module to `geonodeharvester`

Hoping to make lgtm happy

* Add migrations to rename geonode harvester module name

* Improve support for harvesting raster datasets and disable harvesting maps

Co-authored-by: Giovanni Allegri <[email protected]>
Co-authored-by: allyoucanmap <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Toni <[email protected]>
Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>
Co-authored-by: Florian Hoedt <[email protected]>
Co-authored-by: Mohammed Y. Alnajdi <[email protected]>
Co-authored-by: biegan <[email protected]>
Co-authored-by: Ricardo Garcia Silva <[email protected]>
@afabiani afabiani added this to the 4.0.0 milestone Sep 29, 2021
ricardogsilva pushed a commit to ricardogsilva/geonode that referenced this issue Sep 30, 2021
ricardogsilva pushed a commit to ricardogsilva/geonode that referenced this issue Sep 30, 2021
afabiani added a commit that referenced this issue Oct 1, 2021
* Bump urllib3 from 1.26.2 to 1.26.3 (#6908)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.2 to 1.26.3.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/1.26.3/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.2...1.26.3)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Toni <[email protected]>

* [Fixes #6880] Circle CI upload tests fail irregulary (#6881)

* [Fixes #6880] Circle CI upload tests fail irregulary

* CircleCI test fix: sometimes expires due to upload timeout in the test environment

* - Avoid infinite loop on upload testing

* Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment"

This reverts commit 66139fd.

Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>

* [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915)

* Added malnajdi as contributor

* [Fixes #6910] meaningful filename for document download (#6911)

* get meaningful document filenames on download

* - Strip extension from document title before slugify it (e.g.: image.jpg instead of imagejpg.jpg)

Co-authored-by: afabiani <[email protected]>
Co-authored-by: Alessio Fabiani <[email protected]>

* - CircleCI Upload Tests: trying to reduce more the risk of infinite loop on "wait_for_progress"

* [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION (#6913)

* [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION

* [Fixes #6916] unit test for GEOSERVER_LOCATION

* Bump django-cors-headers from 3.6.0 to 3.7.0 (#6901)

Bumps [django-cors-headers](https://github.com/adamchainz/django-cors-headers) from 3.6.0 to 3.7.0.
- [Release notes](https://github.com/adamchainz/django-cors-headers/releases)
- [Changelog](https://github.com/adamchainz/django-cors-headers/blob/master/HISTORY.rst)
- [Commits](adamchainz/django-cors-headers@3.6.0...3.7.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump amqp from 5.0.3 to 5.0.5 (#6905)

Bumps [amqp](https://github.com/celery/py-amqp) from 5.0.3 to 5.0.5.
- [Release notes](https://github.com/celery/py-amqp/releases)
- [Changelog](https://github.com/celery/py-amqp/blob/master/Changelog)
- [Commits](celery/py-amqp@v5.0.3...v5.0.5)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pip from 21.0 to 21.0.1 (#6900)

Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst)
- [Commits](pypa/pip@21.0...21.0.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump coverage from 5.3.1 to 5.4 (#6903)

Bumps [coverage](https://github.com/nedbat/coveragepy) from 5.3.1 to 5.4.
- [Release notes](https://github.com/nedbat/coveragepy/releases)
- [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst)
- [Commits](nedbat/coveragepy@coverage-5.3.1...coverage-5.4)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pytest from 6.2.1 to 6.2.2 (#6907)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 6.2.1 to 6.2.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/master/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@6.2.1...6.2.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump djangorestframework-gis from 0.16 to 0.17 (#6902)

Bumps [djangorestframework-gis](https://github.com/openwisp/django-rest-framework-gis) from 0.16 to 0.17.
- [Release notes](https://github.com/openwisp/django-rest-framework-gis/releases)
- [Changelog](https://github.com/openwisp/django-rest-framework-gis/blob/master/CHANGES.rst)
- [Commits](openwisp/django-rest-framework-gis@v0.16.0...v0.17.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* - Algin setup.cfg to requirements.txt

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923)

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded

* - Add REST APIs test suite to CircleCI

* [Fixes #6918] Removal of QGIS support (#6919)

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* - Fix LGTM issues

* allow Basic authenticated requests in LOCKDOWN mode

* fix to avoid circular import

* flake8 check fix

* added tests

* [Fixes #6880] Circle CI upload tests fail irregulary (#6881)

* [Fixes #6880] Circle CI upload tests fail irregulary

* CircleCI test fix: sometimes expires due to upload timeout in the test environment

* - Avoid infinite loop on upload testing

* Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment"

This reverts commit 66139fd.

Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>

* [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915)

* Added malnajdi as contributor

* Bump pip from 21.0 to 21.0.1 (#6900)

Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst)
- [Commits](pypa/pip@21.0...21.0.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* - Algin setup.cfg to requirements.txt

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923)

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded

* - Add REST APIs test suite to CircleCI

* [Fixes #6918] Removal of QGIS support (#6919)

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* - Fix LGTM issues

* allow Basic authenticated requests in LOCKDOWN mode

* fix to avoid circular import

* - Align to upstream master branch

* [Fixes #7945] Ingest harvested layer data to geonode

* Improve harvesting session and the admin

* fix migration files conflict

* Initial work for implementing stoppable harvesting sessions

* Implement aborting of harvesting celery tasks

Add the `AsynchronousHarvestingSession` model, which is used
to implement sessions for both refreshing of a harvester's
harvestable resources and for the harvesting of remote
resources.

Refactor the `admin`, `api` and `tasks` to use this new model.

Moved some functions out of `harvesting.utils` module in order
to avoid circular imports

* fix tests

* fix conflicts

* Remove accidental duplication of code that crept in during conflict resolution

* Add a couple more tests

* Update signature of get_resource method in WMS harvester worker

* Uncomment `settings.py` line that designates the GeoNode test runner as the one to be used

Co-authored-by: Giovanni Allegri <[email protected]>
Co-authored-by: allyoucanmap <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Toni <[email protected]>
Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>
Co-authored-by: Florian Hoedt <[email protected]>
Co-authored-by: Mohammed Y. Alnajdi <[email protected]>
Co-authored-by: biegan <[email protected]>
Co-authored-by: meomancer <[email protected]>
ricardogsilva pushed a commit to ricardogsilva/geonode that referenced this issue Oct 1, 2021
afabiani pushed a commit that referenced this issue Oct 8, 2021
* Bump urllib3 from 1.26.2 to 1.26.3 (#6908)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.2 to 1.26.3.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/1.26.3/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.2...1.26.3)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Toni <[email protected]>

* [Fixes #6880] Circle CI upload tests fail irregulary (#6881)

* [Fixes #6880] Circle CI upload tests fail irregulary

* CircleCI test fix: sometimes expires due to upload timeout in the test environment

* - Avoid infinite loop on upload testing

* Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment"

This reverts commit 66139fd.

Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>

* [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915)

* Added malnajdi as contributor

* [Fixes #6910] meaningful filename for document download (#6911)

* get meaningful document filenames on download

* - Strip extension from document title before slugify it (e.g.: image.jpg instead of imagejpg.jpg)

Co-authored-by: afabiani <[email protected]>
Co-authored-by: Alessio Fabiani <[email protected]>

* - CircleCI Upload Tests: trying to reduce more the risk of infinite loop on "wait_for_progress"

* [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION (#6913)

* [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION

* [Fixes #6916] unit test for GEOSERVER_LOCATION

* Bump django-cors-headers from 3.6.0 to 3.7.0 (#6901)

Bumps [django-cors-headers](https://github.com/adamchainz/django-cors-headers) from 3.6.0 to 3.7.0.
- [Release notes](https://github.com/adamchainz/django-cors-headers/releases)
- [Changelog](https://github.com/adamchainz/django-cors-headers/blob/master/HISTORY.rst)
- [Commits](adamchainz/django-cors-headers@3.6.0...3.7.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump amqp from 5.0.3 to 5.0.5 (#6905)

Bumps [amqp](https://github.com/celery/py-amqp) from 5.0.3 to 5.0.5.
- [Release notes](https://github.com/celery/py-amqp/releases)
- [Changelog](https://github.com/celery/py-amqp/blob/master/Changelog)
- [Commits](celery/py-amqp@v5.0.3...v5.0.5)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pip from 21.0 to 21.0.1 (#6900)

Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst)
- [Commits](pypa/pip@21.0...21.0.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump coverage from 5.3.1 to 5.4 (#6903)

Bumps [coverage](https://github.com/nedbat/coveragepy) from 5.3.1 to 5.4.
- [Release notes](https://github.com/nedbat/coveragepy/releases)
- [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst)
- [Commits](nedbat/coveragepy@coverage-5.3.1...coverage-5.4)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pytest from 6.2.1 to 6.2.2 (#6907)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 6.2.1 to 6.2.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/master/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@6.2.1...6.2.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump djangorestframework-gis from 0.16 to 0.17 (#6902)

Bumps [djangorestframework-gis](https://github.com/openwisp/django-rest-framework-gis) from 0.16 to 0.17.
- [Release notes](https://github.com/openwisp/django-rest-framework-gis/releases)
- [Changelog](https://github.com/openwisp/django-rest-framework-gis/blob/master/CHANGES.rst)
- [Commits](openwisp/django-rest-framework-gis@v0.16.0...v0.17.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* - Algin setup.cfg to requirements.txt

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923)

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded

* - Add REST APIs test suite to CircleCI

* [Fixes #6918] Removal of QGIS support (#6919)

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* - Fix LGTM issues

* allow Basic authenticated requests in LOCKDOWN mode

* fix to avoid circular import

* flake8 check fix

* added tests

* [Fixes #6880] Circle CI upload tests fail irregulary (#6881)

* [Fixes #6880] Circle CI upload tests fail irregulary

* CircleCI test fix: sometimes expires due to upload timeout in the test environment

* - Avoid infinite loop on upload testing

* Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment"

This reverts commit 66139fd.

Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>

* [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915)

* Added malnajdi as contributor

* Bump pip from 21.0 to 21.0.1 (#6900)

Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst)
- [Commits](pypa/pip@21.0...21.0.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* - Algin setup.cfg to requirements.txt

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923)

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded

* - Add REST APIs test suite to CircleCI

* [Fixes #6918] Removal of QGIS support (#6919)

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* - Fix LGTM issues

* allow Basic authenticated requests in LOCKDOWN mode

* fix to avoid circular import

* - Align to upstream master branch

* [Fixes #7945] Ingest harvested layer data to geonode

* Refactor celery tasks in order to allow performing harvesting of selected harvestable resources

* Implement copying of remote resources for the GeoNode legacy harvester

* Improve harvesting session and the admin

* Ensure `extension` is present for harvested GeoNode documents

* Add missing default attributes to harvested resources

* fixing tests

* fixing tests

* [Fixes #8148] [Harvesting] Adapt / extend remote service model

* - Make sure we set all the "Remote" attributes correctly for a Dataset

* - Getting rid of outdated comments

* fix migration files conflict

* [LGTM] Fix Issues

* [CircleCI] Test Fixes

* Renamed `geonode` harvester module to `geonodeharvester`

Hoping to make lgtm happy

* Add migrations to rename geonode harvester module name

* Merge branch '7945-harvest-layer-data' of https://github.com/meomancer/geonode into ISSUE_8148

# Conflicts:
#	geonode/harvesting/harvesters/base.py

* Merge branch '7945-harvest-layer-data' of https://github.com/meomancer/geonode into ISSUE_8148

# Conflicts:
#	geonode/harvesting/harvesters/base.py

* [Flake8] Autopep8

* - Fixing migrations

* [Dependencies] Align "setup.cfg" to "requirements.txt"

* [Remote Services] Harvest a single resource instead of triggering all the marked ones again.

* - Few more sanity checks on the 'remote_service' property existence

* [Dependencies] Align "setup.cfg" to "requirements.txt"

* - Make sure we don't save a dataset twice in case the "final" upload step has been called again

* - Using "GeonodeLegacyHarvester" on "Remote Services" of type "GN_WMS"

* [Fixes #7945] Ingest harvested layer data to geonode

* Refactor celery tasks in order to allow performing harvesting of selected harvestable resources

* Implement copying of remote resources for the GeoNode legacy harvester

* Improve harvesting session and the admin

* Ensure `extension` is present for harvested GeoNode documents

* Add missing default attributes to harvested resources

* fixing tests

* fixing tests

* fix migration files conflict

* Renamed `geonode` harvester module to `geonodeharvester`

Hoping to make lgtm happy

* Add migrations to rename geonode harvester module name

* Improve support for harvesting raster datasets and disable harvesting maps

* - Make sure ll_bbox and default links have been generated for the geonode resources

* [Harvesters - WMS] Harvesters should merge remote url parameters if specified Harvester

 [Harvesters - WMS] Harvesters should use WMS operations to fetch the GetCapabilities Harvester
 [Harvesters - WMS] GeoNode resources are not correctly defined Harvester
 [Harvesters - WMS] The GeoNode-WMS service legacy type is missing Harvester
 [Harvester] GeoNode Legacy Harvester should fetch the metadata and thumbnail from the Remote resources

* [Fixes #7945] Ingest harvested layer data to geonode

* Improve harvesting session and the admin

* fix migration files conflict

* Initial work for implementing stoppable harvesting sessions

* Implement aborting of harvesting celery tasks

Add the `AsynchronousHarvestingSession` model, which is used
to implement sessions for both refreshing of a harvester's
harvestable resources and for the harvesting of remote
resources.

Refactor the `admin`, `api` and `tasks` to use this new model.

Moved some functions out of `harvesting.utils` module in order
to avoid circular imports

* fix tests

* fix conflicts

* Remove accidental duplication of code that crept in during conflict resolution

* Add a couple more tests

* Update signature of get_resource method in WMS harvester worker

* Uncomment `settings.py` line that designates the GeoNode test runner as the one to be used

* - Tests Fixes

* - Adapt to the new "AsyncHarvestingSession" model changes

* - Better management of "Refreshing" and "Harvesting" sessions status from the Remote Service detail page

* - Tests Fixes

* - General improvements to the PR and fixes to the migrations

* - Cleaning up models and getting rid of unused proxy stuff

* - Tests fixes

* - "upgradeservices" management command

* fix filter icon

* - Fixes and improvements to the harvester WMS resource discovery

* - Tests fixes

* - Create thumbs on geonode resources from WMS harvesters

* - Update harvester on "rescan" service action

* fix filtering with TableFilter

* fix filtering views and html

* - Create thumbs on geonode resources from WMS harvesters

* fix unefined checkbox id

* - Create thumbs on geonode resources from WMS harvesters

* - Remote Services: create thumbnail uses the original SRID

* [Pep8] Fixing Flake8 issues

* - Make sure the "thumb" is shown only after the resource has been fully uploaded

* - WMS Harvester: make sure we parse the correct CRS and BBOX from GetCapa XML response

* - Fix thumbnail generation through the APIs

(cherry picked from commit 1c07928)

# Conflicts:
#	geonode/base/api/views.py

Co-authored-by: Giovanni Allegri <[email protected]>
Co-authored-by: allyoucanmap <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Toni <[email protected]>
Co-authored-by: Florian Hoedt <[email protected]>
Co-authored-by: Mohammed Y. Alnajdi <[email protected]>
Co-authored-by: biegan <[email protected]>
Co-authored-by: Ricardo Garcia Silva <[email protected]>
Co-authored-by: meomancer <[email protected]>
Co-authored-by: Ricardo Garcia Silva <[email protected]>
afabiani added a commit that referenced this issue Oct 15, 2021
* Bump urllib3 from 1.26.2 to 1.26.3 (#6908)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.2 to 1.26.3.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/1.26.3/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.2...1.26.3)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Toni <[email protected]>

* [Fixes #6880] Circle CI upload tests fail irregulary (#6881)

* [Fixes #6880] Circle CI upload tests fail irregulary

* CircleCI test fix: sometimes expires due to upload timeout in the test environment

* - Avoid infinite loop on upload testing

* Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment"

This reverts commit 66139fd.

Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>

* [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915)

* Added malnajdi as contributor

* [Fixes #6910] meaningful filename for document download (#6911)

* get meaningful document filenames on download

* - Strip extension from document title before slugify it (e.g.: image.jpg instead of imagejpg.jpg)

Co-authored-by: afabiani <[email protected]>
Co-authored-by: Alessio Fabiani <[email protected]>

* - CircleCI Upload Tests: trying to reduce more the risk of infinite loop on "wait_for_progress"

* [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION (#6913)

* [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION

* [Fixes #6916] unit test for GEOSERVER_LOCATION

* Bump django-cors-headers from 3.6.0 to 3.7.0 (#6901)

Bumps [django-cors-headers](https://github.com/adamchainz/django-cors-headers) from 3.6.0 to 3.7.0.
- [Release notes](https://github.com/adamchainz/django-cors-headers/releases)
- [Changelog](https://github.com/adamchainz/django-cors-headers/blob/master/HISTORY.rst)
- [Commits](adamchainz/django-cors-headers@3.6.0...3.7.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump amqp from 5.0.3 to 5.0.5 (#6905)

Bumps [amqp](https://github.com/celery/py-amqp) from 5.0.3 to 5.0.5.
- [Release notes](https://github.com/celery/py-amqp/releases)
- [Changelog](https://github.com/celery/py-amqp/blob/master/Changelog)
- [Commits](celery/py-amqp@v5.0.3...v5.0.5)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pip from 21.0 to 21.0.1 (#6900)

Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst)
- [Commits](pypa/pip@21.0...21.0.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump coverage from 5.3.1 to 5.4 (#6903)

Bumps [coverage](https://github.com/nedbat/coveragepy) from 5.3.1 to 5.4.
- [Release notes](https://github.com/nedbat/coveragepy/releases)
- [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst)
- [Commits](nedbat/coveragepy@coverage-5.3.1...coverage-5.4)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pytest from 6.2.1 to 6.2.2 (#6907)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 6.2.1 to 6.2.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/master/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@6.2.1...6.2.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump djangorestframework-gis from 0.16 to 0.17 (#6902)

Bumps [djangorestframework-gis](https://github.com/openwisp/django-rest-framework-gis) from 0.16 to 0.17.
- [Release notes](https://github.com/openwisp/django-rest-framework-gis/releases)
- [Changelog](https://github.com/openwisp/django-rest-framework-gis/blob/master/CHANGES.rst)
- [Commits](openwisp/django-rest-framework-gis@v0.16.0...v0.17.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* - Algin setup.cfg to requirements.txt

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923)

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded

* - Add REST APIs test suite to CircleCI

* [Fixes #6918] Removal of QGIS support (#6919)

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* - Fix LGTM issues

* allow Basic authenticated requests in LOCKDOWN mode

* fix to avoid circular import

* flake8 check fix

* added tests

* [Fixes #6880] Circle CI upload tests fail irregulary (#6881)

* [Fixes #6880] Circle CI upload tests fail irregulary

* CircleCI test fix: sometimes expires due to upload timeout in the test environment

* - Avoid infinite loop on upload testing

* Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment"

This reverts commit 66139fd.

Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>

* [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915)

* Added malnajdi as contributor

* Bump pip from 21.0 to 21.0.1 (#6900)

Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst)
- [Commits](pypa/pip@21.0...21.0.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* - Algin setup.cfg to requirements.txt

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923)

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded

* - Add REST APIs test suite to CircleCI

* [Fixes #6918] Removal of QGIS support (#6919)

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* - Fix LGTM issues

* allow Basic authenticated requests in LOCKDOWN mode

* fix to avoid circular import

* - Align to upstream master branch

* [Fixes #7945] Ingest harvested layer data to geonode

* Improve harvesting session and the admin

* fix migration files conflict

* Initial work for implementing stoppable harvesting sessions

* Implement aborting of harvesting celery tasks

Add the `AsynchronousHarvestingSession` model, which is used
to implement sessions for both refreshing of a harvester's
harvestable resources and for the harvesting of remote
resources.

Refactor the `admin`, `api` and `tasks` to use this new model.

Moved some functions out of `harvesting.utils` module in order
to avoid circular imports

* fix tests

* fix conflicts

* Remove accidental duplication of code that crept in during conflict resolution

* Add a couple more tests

* Update signature of get_resource method in WMS harvester worker

* Add harvesting scheduler task

* Improving support for the harvesting scheduler task

* Continue implementation of the harvesting scheduler task

* Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore

* Uncomment `settings.py` line that designates the GeoNode test runner as the one to be used

* Add harvesting scheduler task

* Improving support for the harvesting scheduler task

* Continue implementation of the harvesting scheduler task

* Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore

* Revert "Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore"

This reverts commit 8d50cf2.

* Remove reference to the django-celery-beat app in the harvesting models and signals

* Implement different strategy for the harvesting scheduler

- remove django-celery-beat as a dependency
- implement a simple harvesting scheduler as a celery task
- implement an action for resetting a harvester's status
- add (and fix) tests

* [Fixes #7945] Ingest harvested layer data to geonode

* Improve harvesting session and the admin

* fix migration files conflict

* Initial work for implementing stoppable harvesting sessions

* Implement aborting of harvesting celery tasks

Add the `AsynchronousHarvestingSession` model, which is used
to implement sessions for both refreshing of a harvester's
harvestable resources and for the harvesting of remote
resources.

Refactor the `admin`, `api` and `tasks` to use this new model.

Moved some functions out of `harvesting.utils` module in order
to avoid circular imports

* fix tests

* fix conflicts

* Remove accidental duplication of code that crept in during conflict resolution

* Add a couple more tests

* Uncomment `settings.py` line that designates the GeoNode test runner as the one to be used

* Add harvesting scheduler task

* Improving support for the harvesting scheduler task

* Continue implementation of the harvesting scheduler task

* Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore

* Revert "Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore"

This reverts commit 8d50cf2.

* Remove reference to the django-celery-beat app in the harvesting models and signals

* Implement different strategy for the harvesting scheduler

- remove django-celery-beat as a dependency
- implement a simple harvesting scheduler as a celery task
- implement an action for resetting a harvester's status
- add (and fix) tests

* Add harvesting scheduler task

* Improving support for the harvesting scheduler task

* Continue implementation of the harvesting scheduler task

* Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore

* fix migration conflicts

* Fix broken pavement.py

* Fix invalid tiemporal computation after creating a new harvester

* Add more tests

* - Just making the tasks initialization coherent with the other modules

Co-authored-by: Giovanni Allegri <[email protected]>
Co-authored-by: allyoucanmap <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Toni <[email protected]>
Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>
Co-authored-by: Florian Hoedt <[email protected]>
Co-authored-by: Mohammed Y. Alnajdi <[email protected]>
Co-authored-by: biegan <[email protected]>
Co-authored-by: meomancer <[email protected]>
ricardogsilva pushed a commit to ricardogsilva/geonode that referenced this issue Oct 19, 2021
ricardogsilva pushed a commit to ricardogsilva/geonode that referenced this issue Oct 19, 2021
ricardogsilva pushed a commit to ricardogsilva/geonode that referenced this issue Oct 21, 2021
afabiani added a commit that referenced this issue Oct 25, 2021
* Bump urllib3 from 1.26.2 to 1.26.3 (#6908)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.2 to 1.26.3.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/1.26.3/CHANGES.rst)
- [Commits](urllib3/urllib3@1.26.2...1.26.3)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Toni <[email protected]>

* [Fixes #6880] Circle CI upload tests fail irregulary (#6881)

* [Fixes #6880] Circle CI upload tests fail irregulary

* CircleCI test fix: sometimes expires due to upload timeout in the test environment

* - Avoid infinite loop on upload testing

* Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment"

This reverts commit 66139fd.

Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>

* [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915)

* Added malnajdi as contributor

* [Fixes #6910] meaningful filename for document download (#6911)

* get meaningful document filenames on download

* - Strip extension from document title before slugify it (e.g.: image.jpg instead of imagejpg.jpg)

Co-authored-by: afabiani <[email protected]>
Co-authored-by: Alessio Fabiani <[email protected]>

* - CircleCI Upload Tests: trying to reduce more the risk of infinite loop on "wait_for_progress"

* [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION (#6913)

* [Fixes #6916] gsimporter.api.NotFound caused by missing trailing slash at the end of GEOSERVER_LOCATION

* [Fixes #6916] unit test for GEOSERVER_LOCATION

* Bump django-cors-headers from 3.6.0 to 3.7.0 (#6901)

Bumps [django-cors-headers](https://github.com/adamchainz/django-cors-headers) from 3.6.0 to 3.7.0.
- [Release notes](https://github.com/adamchainz/django-cors-headers/releases)
- [Changelog](https://github.com/adamchainz/django-cors-headers/blob/master/HISTORY.rst)
- [Commits](adamchainz/django-cors-headers@3.6.0...3.7.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump amqp from 5.0.3 to 5.0.5 (#6905)

Bumps [amqp](https://github.com/celery/py-amqp) from 5.0.3 to 5.0.5.
- [Release notes](https://github.com/celery/py-amqp/releases)
- [Changelog](https://github.com/celery/py-amqp/blob/master/Changelog)
- [Commits](celery/py-amqp@v5.0.3...v5.0.5)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pip from 21.0 to 21.0.1 (#6900)

Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst)
- [Commits](pypa/pip@21.0...21.0.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump coverage from 5.3.1 to 5.4 (#6903)

Bumps [coverage](https://github.com/nedbat/coveragepy) from 5.3.1 to 5.4.
- [Release notes](https://github.com/nedbat/coveragepy/releases)
- [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst)
- [Commits](nedbat/coveragepy@coverage-5.3.1...coverage-5.4)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pytest from 6.2.1 to 6.2.2 (#6907)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 6.2.1 to 6.2.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/master/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@6.2.1...6.2.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump djangorestframework-gis from 0.16 to 0.17 (#6902)

Bumps [djangorestframework-gis](https://github.com/openwisp/django-rest-framework-gis) from 0.16 to 0.17.
- [Release notes](https://github.com/openwisp/django-rest-framework-gis/releases)
- [Changelog](https://github.com/openwisp/django-rest-framework-gis/blob/master/CHANGES.rst)
- [Commits](openwisp/django-rest-framework-gis@v0.16.0...v0.17.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* - Algin setup.cfg to requirements.txt

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923)

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded

* - Add REST APIs test suite to CircleCI

* [Fixes #6918] Removal of QGIS support (#6919)

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* - Fix LGTM issues

* allow Basic authenticated requests in LOCKDOWN mode

* fix to avoid circular import

* flake8 check fix

* added tests

* [Fixes #6880] Circle CI upload tests fail irregulary (#6881)

* [Fixes #6880] Circle CI upload tests fail irregulary

* CircleCI test fix: sometimes expires due to upload timeout in the test environment

* - Avoid infinite loop on upload testing

* Revert "CircleCI test fix: sometimes expires due to upload timeout in the test environment"

This reverts commit 66139fd.

Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>

* [Fixes #6914] Remove "add to basket" tool for documents and maps (#6915)

* Added malnajdi as contributor

* Bump pip from 21.0 to 21.0.1 (#6900)

Bumps [pip](https://github.com/pypa/pip) from 21.0 to 21.0.1.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/master/NEWS.rst)
- [Commits](pypa/pip@21.0...21.0.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* - Algin setup.cfg to requirements.txt

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has… (#6923)

* [Fixes #6922][REST API v2] Expose the curated thumbnail URL if it has been uploaded

* - Add REST APIs test suite to CircleCI

* [Fixes #6918] Removal of QGIS support (#6919)

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* [Cleanup and Refactor] Remove QGIS server backend dependencies

* - Fix LGTM issues

* allow Basic authenticated requests in LOCKDOWN mode

* fix to avoid circular import

* - Align to upstream master branch

* [Fixes #7945] Ingest harvested layer data to geonode

* Improve harvesting session and the admin

* fix migration files conflict

* Initial work for implementing stoppable harvesting sessions

* Implement aborting of harvesting celery tasks

Add the `AsynchronousHarvestingSession` model, which is used
to implement sessions for both refreshing of a harvester's
harvestable resources and for the harvesting of remote
resources.

Refactor the `admin`, `api` and `tasks` to use this new model.

Moved some functions out of `harvesting.utils` module in order
to avoid circular imports

* fix tests

* fix conflicts

* Add a couple more tests

* Uncomment `settings.py` line that designates the GeoNode test runner as the one to be used

* Add harvesting scheduler task

* Improving support for the harvesting scheduler task

* Continue implementation of the harvesting scheduler task

* Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore

* Revert "Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore"

This reverts commit 8d50cf2.

* Remove reference to the django-celery-beat app in the harvesting models and signals

* Implement different strategy for the harvesting scheduler

- remove django-celery-beat as a dependency
- implement a simple harvesting scheduler as a celery task
- implement an action for resetting a harvester's status
- add (and fix) tests

* Add harvesting scheduler task

* Improving support for the harvesting scheduler task

* Continue implementation of the harvesting scheduler task

* Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore

* [Fixes #7945] Ingest harvested layer data to geonode

* Improve harvesting session and the admin

* fix migration files conflict

* Implement aborting of harvesting celery tasks

Add the `AsynchronousHarvestingSession` model, which is used
to implement sessions for both refreshing of a harvester's
harvestable resources and for the harvesting of remote
resources.

Refactor the `admin`, `api` and `tasks` to use this new model.

Moved some functions out of `harvesting.utils` module in order
to avoid circular imports

* fix tests

* fix conflicts

* Uncomment `settings.py` line that designates the GeoNode test runner as the one to be used

* Add harvesting scheduler task

* Improving support for the harvesting scheduler task

* Revert "Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore"

This reverts commit 8d50cf2.

* Remove reference to the django-celery-beat app in the harvesting models and signals

* Implement different strategy for the harvesting scheduler

- remove django-celery-beat as a dependency
- implement a simple harvesting scheduler as a celery task
- implement an action for resetting a harvester's status
- add (and fix) tests

* Improving support for the harvesting scheduler task

* Continue implementation of the harvesting scheduler task

* Begin implementation of a harvester for current GeoNode version

* Continue implementation of a modern harvester for remote GeoNode deployments

* Rebase and fix migration conflicts

* Fix invalid tiemporal computation after creating a new harvester

* fixing pagination (wip)

* Fix bugs with remote harvested layers

* Try out both geonode harvester worker classes and iron out inconsistencies

* Remove commented line

* Fix tests

* Harvester API serializer no longer tries to update harvestable resources upon creation.

This makes it behave in a similar way as when a harvester is created through the django admin

* Made GeonodeUnifiedHarvesterWorker the default harvester for geonode remotes

* Fix bits that were incorrectly merged when rebasing

* fix flake8 errors

* [Fixes #7945] Ingest harvested layer data to geonode

* Improve harvesting session and the admin

* fix migration files conflict

* Initial work for implementing stoppable harvesting sessions

* Implement aborting of harvesting celery tasks

Add the `AsynchronousHarvestingSession` model, which is used
to implement sessions for both refreshing of a harvester's
harvestable resources and for the harvesting of remote
resources.

Refactor the `admin`, `api` and `tasks` to use this new model.

Moved some functions out of `harvesting.utils` module in order
to avoid circular imports

* fix tests

* fix conflicts

* Remove accidental duplication of code that crept in during conflict resolution

* Add a couple more tests

* Uncomment `settings.py` line that designates the GeoNode test runner as the one to be used

* Add harvesting scheduler task

* Improving support for the harvesting scheduler task

* Continue implementation of the harvesting scheduler task

* Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore

* Revert "Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore"

This reverts commit 8d50cf2.

* Remove reference to the django-celery-beat app in the harvesting models and signals

* Implement different strategy for the harvesting scheduler

- remove django-celery-beat as a dependency
- implement a simple harvesting scheduler as a celery task
- implement an action for resetting a harvester's status
- add (and fix) tests

* Add harvesting scheduler task

* Continue implementation of the harvesting scheduler task

* Improve harvesting session and the admin

* Initial work for implementing stoppable harvesting sessions

* Implement aborting of harvesting celery tasks

Add the `AsynchronousHarvestingSession` model, which is used
to implement sessions for both refreshing of a harvester's
harvestable resources and for the harvesting of remote
resources.

Refactor the `admin`, `api` and `tasks` to use this new model.

Moved some functions out of `harvesting.utils` module in order
to avoid circular imports

* fix tests

* Update signature of get_resource method in WMS harvester worker

* Add harvesting scheduler task

* Improving support for the harvesting scheduler task

* Continue implementation of the harvesting scheduler task

* Revert "Remove the dependency on `django-celery-beat`, as the harvesting does not need it anymore"

This reverts commit 8d50cf2.

* Remove reference to the django-celery-beat app in the harvesting models and signals

* Implement different strategy for the harvesting scheduler

- remove django-celery-beat as a dependency
- implement a simple harvesting scheduler as a celery task
- implement an action for resetting a harvester's status
- add (and fix) tests

* Improving support for the harvesting scheduler task

* fix migration conflicts

* Begin implementation of ArcGIS-related harvester

* Continue implementation of ArcGIS-related harvester

* Continue implementation of arcgis harvester

* Continue implementation of arcgis harvester

* HAndling harvesting of MapServer layers

* Move gxp enumerations from the services package to the layers one

* Modify `dataset_detail()` view in order to configure gxp layer for layers that come from ArcGIS REST services

* Conclude implementation of ArcGIS harvester

* Add some tests

* Update code

* Implement resource_name_filter for ESRI harvester

* Minor changes to keep compatibility with Remote Services

* [ArcGIS Harvester] Make sure we can parse also nested services (e.g.: https://pro-ags2.dfs.un.org/arcgis/rest/services/UNMISS/UNMISS_Road_Rehabilitation/MapServer)

* [Remote Services] Re-enable ArcGIS Remote Services

* Improve WMS harvester based on recent develpments

* - Better management of the "ll_bbox" and "thumbnail" at the "ResourceManager" level

* Remove `name` from the default geonode parameters generated by the base harvester

This was a regression. Since GeoNode documents do not have a name, this field must only be added when relevant (e.g. when creating datasets)

* [CircleCI] Fix test cases

Co-authored-by: Giovanni Allegri <[email protected]>
Co-authored-by: allyoucanmap <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Toni <[email protected]>
Co-authored-by: Alessio Fabiani <[email protected]>
Co-authored-by: afabiani <[email protected]>
Co-authored-by: Florian Hoedt <[email protected]>
Co-authored-by: Mohammed Y. Alnajdi <[email protected]>
Co-authored-by: biegan <[email protected]>
Co-authored-by: meomancer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants