diff --git a/CONTRIBUTORS b/CONTRIBUTORS index ba50a8c6..03f26522 100644 --- a/CONTRIBUTORS +++ b/CONTRIBUTORS @@ -2,8 +2,8 @@ Contributors ordered by number of commits: ========================================== Manuel Giffels Max Fischer -Alexander Haas Stefan Kroboth +Alexander Haas Eileen Kuehn matthias.schnepf ubdsv diff --git a/docs/source/adapters/site.rst b/docs/source/adapters/site.rst index 92f30e9e..a1529d8d 100644 --- a/docs/source/adapters/site.rst +++ b/docs/source/adapters/site.rst @@ -199,11 +199,11 @@ Available adapter configuration options | Option | Short Description | Requirement | +================+===================================================================================+=================+ | max_age | The result of the `condor_status` call is cached for `max_age` in minutes. | **Required** | - +================+===================================================================================+=================+ + +----------------+-----------------------------------------------------------------------------------+-----------------+ | bulk_size | Maximum number of jobs to handle per bulk invocation of a condor tool. | **Optional** | + + + + | | Default: 100 | | - +================+===================================================================================+=================+ + +----------------+-----------------------------------------------------------------------------------+-----------------+ | bulk_delay | Maximum duration in seconds to wait per bulk invocation of a condor tool. | **Optional** | + + + + | | Default: 1.0 | | diff --git a/docs/source/api/tardis.rest.token_generator.generate_token.rst b/docs/source/api/tardis.rest.token_generator.generate_token.rst deleted file mode 100644 index 42e4a004..00000000 --- a/docs/source/api/tardis.rest.token_generator.generate_token.rst +++ /dev/null @@ -1,7 +0,0 @@ -tardis.rest.token\_generator.generate\_token module -=================================================== - -.. automodule:: tardis.rest.token_generator.generate_token - :members: - :undoc-members: - :show-inheritance: diff --git a/docs/source/api/tardis.rest.token_generator.rst b/docs/source/api/tardis.rest.token_generator.rst deleted file mode 100644 index 4e5ae29c..00000000 --- a/docs/source/api/tardis.rest.token_generator.rst +++ /dev/null @@ -1,15 +0,0 @@ -tardis.rest.token\_generator package -==================================== - -.. automodule:: tardis.rest.token_generator - :members: - :undoc-members: - :show-inheritance: - -Submodules ----------- - -.. toctree:: - :maxdepth: 4 - - tardis.rest.token_generator.generate_token diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index 9975d86d..25b2dbee 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -1,4 +1,4 @@ -.. Created by changelog.py at 2023-02-23, command +.. Created by changelog.py at 2023-02-24, command '/Users/giffler/.cache/pre-commit/repor6pnmwlm/py_env-python3.10/bin/changelog docs/source/changes compile --output=docs/source/changelog.rst' based on the format of 'https://keepachangelog.com/' @@ -6,13 +6,14 @@ CHANGELOG ######### -[Unreleased] - 2023-02-23 -========================= +[0.7.0] - 2023-02-24 +==================== Added ----- * Introduce a TARDIS REST API to query the state of resources from SqlRegistry +* Ensure python3.10 compatibility * Added support for manual draining of drones using the REST API * Add support for passing environment variables as executable arguments to support HTCondor grid universe * Added support for application credentials of the OpenStack site adapter @@ -21,15 +22,23 @@ Added Changed ------- +* Adjust ElasticSearch plugin to support client versions >=7.17,<8.0.0 +* Remove granularity in Standardiser to enable earlier creation of new drones +* Introduced Bulk Executor and HTCondor Bulk Operations * SSHExecutor respects the remote MaxSessions via queueing * Remove minimum core limit (Standardiser) from pool factory * Change drone state initialisation and notification of plugins +* REST API cookie authentication and refactoring +* Adjust Prometheus plugin to the latest aioprometheus version 21.9.0 Fixed ----- * Unique constraints in database schema have been fixed to allow same machine_type and remote_resource_uuid on multiple sites * Update the remote_resource_uuid in sqlite registry on a each update +* REST API does not suppress KeyboardInterrupt +* Fixing recurrent cancellation of jobs TIMEOUTED in Slurm +* Fixed state transition for stopped workers [0.6.0] - 2021-08-09 ==================== diff --git a/docs/source/changes/100.fix_transition_within_cleanupstate.yaml b/docs/source/changes/100.fix_transition_within_cleanupstate.yaml index 5a9a879d..99fdcdd7 100644 --- a/docs/source/changes/100.fix_transition_within_cleanupstate.yaml +++ b/docs/source/changes/100.fix_transition_within_cleanupstate.yaml @@ -1,6 +1,5 @@ category: fixed -summary: "The `CleanupState` is now taking into account the status of the resource\ - \ for state transitions" +summary: "The `CleanupState` is now taking into account the status of the resource for state transitions" description: | The `CleanupState` is now taking into account the status of the resource and two missing state transitions have been added to the transistion dictionary. diff --git a/docs/source/changes/126.fix_config_translation.yaml b/docs/source/changes/126.fix_config_translation.yaml index 000eac96..60314cec 100644 --- a/docs/source/changes/126.fix_config_translation.yaml +++ b/docs/source/changes/126.fix_config_translation.yaml @@ -1,7 +1,8 @@ category: fixed summary: "Fix the translation of cloud init scripts into base64 encoded strings" -description: "The translation of cloud init scripts into base64 encoded strings has\ - \ been fixed. Reason was changing a data structure\nwhile interating over it. \n" +description: | + The translation of cloud init scripts into base64 encoded strings has + been fixed. Reason was changing a data structure while interating over it. pull requests: - 126 version: 0.3.0 diff --git a/docs/source/changes/138_changed_moved_startupcommand_in_machine_type_configuration.yaml b/docs/source/changes/138_changed_moved_startupcommand_in_machine_type_configuration.yaml index 043ed402..5c2f0314 100644 --- a/docs/source/changes/138_changed_moved_startupcommand_in_machine_type_configuration.yaml +++ b/docs/source/changes/138_changed_moved_startupcommand_in_machine_type_configuration.yaml @@ -1,6 +1,5 @@ category: changed -summary: "The SLURM adapter can now be configured to use different startup commands\ - \ for each machine type." +summary: "The SLURM adapter can now be configured to use different startup commands for each machine type." description: | The SLURM adapter can now be configured to use different startup commands for each machine type. The old behaviour of providing one startup command is still supported, but will be deprecated in the next major release 0.4.0. diff --git a/docs/source/changes/139_changed_moved_startupcommand_in_machine_type_configuration_moab.yaml b/docs/source/changes/139_changed_moved_startupcommand_in_machine_type_configuration_moab.yaml index 7481901d..ec39b418 100644 --- a/docs/source/changes/139_changed_moved_startupcommand_in_machine_type_configuration_moab.yaml +++ b/docs/source/changes/139_changed_moved_startupcommand_in_machine_type_configuration_moab.yaml @@ -1,6 +1,5 @@ category: changed -summary: "The Moab adapter can now be configured to use different startup commands\ - \ for each machine type." +summary: "The Moab adapter can now be configured to use different startup commands for each machine type." description: | The Moab adapter can now be configured to use different startup commands for each machine type. The old behaviour of providing one startup command is still supported, but will be deprecated in the next major release 0.4.0. diff --git a/docs/source/changes/145.add_ssh_connection_sharing.yaml b/docs/source/changes/145.add_ssh_connection_sharing.yaml index 060ffc7c..7d131181 100644 --- a/docs/source/changes/145.add_ssh_connection_sharing.yaml +++ b/docs/source/changes/145.add_ssh_connection_sharing.yaml @@ -1,6 +1,5 @@ category: added -summary: "Add ssh connection sharing to `SSHExecutor` in order to re-use existing\ - \ connection" +summary: "Add ssh connection sharing to `SSHExecutor` in order to re-use existing connection" description: | The `SSHExector` is now re-using existing connections. Closed connections are automatically reestablished. This will avoid connection problems when bothering a remote ssh server with too many requests in too short intervals. diff --git a/docs/source/changes/146.improve_logging.yaml b/docs/source/changes/146.improve_logging.yaml index d345cc7b..fed64b63 100644 --- a/docs/source/changes/146.improve_logging.yaml +++ b/docs/source/changes/146.improve_logging.yaml @@ -1,6 +1,5 @@ category: changed -summary: "Added log channels and adjusted log levels according to the conventions\ - \ in `COBalD` documentation" +summary: "Added log channels and adjusted log levels according to the conventions in `COBalD` documentation" description: | Added log channels and adjusted log levels according to the conventions in the `COBalD` documentation. This will improve the user's ability to filter log messages according their needs. diff --git a/docs/source/changes/166.add_drone_heartbeat_interval.yaml b/docs/source/changes/166.add_drone_heartbeat_interval.yaml index 1850869c..7494d9ea 100644 --- a/docs/source/changes/166.add_drone_heartbeat_interval.yaml +++ b/docs/source/changes/166.add_drone_heartbeat_interval.yaml @@ -1,6 +1,5 @@ category: added -summary: "An optional and per site configurable drone heartbeat interval has been\ - \ added" +summary: "An optional and per site configurable drone heartbeat interval has been added" description: | Add an optional and per site configurable drone heartbeat interval to TARDIS. The heartbeat interval is defined as the time between two consecutive calls of the drones run method. The heartbeat interval defaults to 60s. diff --git a/docs/source/changes/169.fix_drone_lifetime.yaml b/docs/source/changes/169.fix_drone_lifetime.yaml index 60ab648b..f84b2eae 100644 --- a/docs/source/changes/169.fix_drone_lifetime.yaml +++ b/docs/source/changes/169.fix_drone_lifetime.yaml @@ -1,6 +1,5 @@ category: fixed -summary: "Fixes a bug that the drone_minimum_lifetime parameter is not working as\ - \ described in the documentation" +summary: "Fixes a bug that the drone_minimum_lifetime parameter is not working as described in the documentation" description: | The `drone_minimum_lifetime` parameter is not working as expected and described in the documentation. `drone_minimum_lifetime` is meant to be a generic site parameter. However the code is trying to look it up in the diff --git a/docs/source/changes/173.fix_meta_data_translation_htcondor.yaml b/docs/source/changes/173.fix_meta_data_translation_htcondor.yaml index 375e7392..2257dda0 100644 --- a/docs/source/changes/173.fix_meta_data_translation_htcondor.yaml +++ b/docs/source/changes/173.fix_meta_data_translation_htcondor.yaml @@ -1,6 +1,5 @@ category: fixed -summary: "Fixes a bug in the HTCondor Site Adapter which leads to wrong requirements\ - \ when using non HTCondor OBS" +summary: "Fixes a bug in the HTCondor Site Adapter which leads to wrong requirements when using non HTCondor OBS" description: | The HTCondor Site Adapter takes a wrong `machine_meta_data_translation_mapping` into account in some circumstances. Due to a bug introduced in #157, the HTCondor Site Adapter uses the `machine_meta_data_translation_mapping` of the diff --git a/docs/source/changes/183.add_rest_api.yaml b/docs/source/changes/183.add_rest_api.yaml index 0cddbc2d..ef75b5fb 100644 --- a/docs/source/changes/183.add_rest_api.yaml +++ b/docs/source/changes/183.add_rest_api.yaml @@ -1,8 +1,10 @@ category: added summary: "Introduce a TARDIS REST API to query the state of resources from SqlRegistry" description: | - Introduction of a REST API to query the state of resources from the SqlRegistry. The REST API is using the FastApi - framework in combination with an uvicorn ASGI server. JSON Web Token and OAuth2 scopes are supported for + Introduction of a REST API to query the state of resources from the + SqlRegistry. The REST API is using the FastApi framework in combination with + an uvicorn ASGI server. JSON Web Token and OAuth2 scopes are supported for authentication and authorization. pull requests: - 183 +version: 0.7.0 diff --git a/docs/source/changes/209.change_resource_granularity.yaml b/docs/source/changes/209.change_resource_granularity.yaml new file mode 100644 index 00000000..68d7449a --- /dev/null +++ b/docs/source/changes/209.change_resource_granularity.yaml @@ -0,0 +1,9 @@ +category: changed +summary: "Remove granularity in Standardiser to enable earlier creation of new drones" +description: | + With granularity new drones are requested when `demand>supply+granularity`. + Remove granularity in Standardiser to enable earlier creation of new drones + when `demand>supply`. +pull requests: +- 209 +version: 0.7.0 diff --git a/docs/source/changes/211.change_support_new_prometheus_releases.yaml b/docs/source/changes/211.change_support_new_prometheus_releases.yaml new file mode 100644 index 00000000..b010d10f --- /dev/null +++ b/docs/source/changes/211.change_support_new_prometheus_releases.yaml @@ -0,0 +1,9 @@ +category: changed +summary: "Adjust Prometheus plugin to the latest aioprometheus version 21.9.0" +description: | + `aioprometheus` has changed its API so that metrics are automatically + registered when they are created. Prometheus plugin has been changed + accordingly and the installation of `aioprometheus>=21.9.0` has been enforced. +pull requests: +- 211 +version: 0.7.0 diff --git a/docs/source/changes/213.add_python_3.10_compatibilty.yaml b/docs/source/changes/213.add_python_3.10_compatibilty.yaml new file mode 100644 index 00000000..7a0e0f0c --- /dev/null +++ b/docs/source/changes/213.add_python_3.10_compatibilty.yaml @@ -0,0 +1,9 @@ +category: added +summary: "Ensure python3.10 compatibility" +description: | + Ensure python3.10 compatibility. Enable Python 3.10 unittests, adds + 3.10 as supported release into setup.py and fixes a few deprecation warnings + occuring when executing unittests under 3.10. +pull requests: +- 213 +version: 0.7.0 diff --git a/docs/source/changes/218.respect_ssh_maxsessions.yaml b/docs/source/changes/218.respect_ssh_maxsessions.yaml index d0bb08db..47728a40 100644 --- a/docs/source/changes/218.respect_ssh_maxsessions.yaml +++ b/docs/source/changes/218.respect_ssh_maxsessions.yaml @@ -4,7 +4,8 @@ description: | The SSHExecutor now is aware of sshd MaxSessions, which is a limit on the concurrent operations per connection. If more operations are to be run at once, operations are queued until a session becomes available. -pull requests: -- 218 issues: - 217 +pull requests: +- 218 +version: 0.7.0 diff --git a/docs/source/changes/220.fix_unique_constraints_db_schema.yaml b/docs/source/changes/220.fix_unique_constraints_db_schema.yaml index 05bcedc0..91b25e5a 100644 --- a/docs/source/changes/220.fix_unique_constraints_db_schema.yaml +++ b/docs/source/changes/220.fix_unique_constraints_db_schema.yaml @@ -3,7 +3,6 @@ summary: "Unique constraints in database schema have been fixed to allow same ma description: | The unique constraints in the datebase schema have been relaxed to allow the same machine_type and the same remote_resource_uuid to be used on multiple sites. In addition, the unittest of the SqliteRegistry have been improved. -pull_requests: - - 220 issues: - - 219 +- 219 +version: 0.7.0 diff --git a/docs/source/changes/224.changed_htcondor_bulk_operations.yaml b/docs/source/changes/224.changed_htcondor_bulk_operations.yaml new file mode 100644 index 00000000..fb3b4b76 --- /dev/null +++ b/docs/source/changes/224.changed_htcondor_bulk_operations.yaml @@ -0,0 +1,13 @@ +category: changed +summary: "Introduced Bulk Executor and HTCondor Bulk Operations" +description: | + Introduced bulk execution to HTCondor SiteAdapter including generic + AsyncBulkCall framework class for collecting tasks to execute in bulk. + HTCondorAdapter uses bulk executions for its commands `deploy resource`, + `stop resource` and `terminate resource`. Changes the Resource UUID + format used by the HTCondor Site adapter to `ClusterId.ProcId`. +issues: +- 223 +pull requests: +- 224 +version: 0.7.0 diff --git a/docs/source/changes/230.change_support_new_es_client.yaml b/docs/source/changes/230.change_support_new_es_client.yaml new file mode 100644 index 00000000..089e71b7 --- /dev/null +++ b/docs/source/changes/230.change_support_new_es_client.yaml @@ -0,0 +1,12 @@ +category: changed +summary: "Adjust ElasticSearch plugin to support client versions >=7.17,<8.0.0" +description: | + The latest versions of the Elasticsearch client have a compatibility + mode which can also be used for newer server versions. Includes also a fix + for the case where `resource_status` is not part of `resource_attributes`, + which occasionally caused crashes. Newer versions of the Elasticsearch client + require the `scheme` parameter to be set. By setting this already, it will + be easier to eventually transition to client version 8. +pull requests: +- 230 +version: 0.7.0 diff --git a/docs/source/changes/234.fixed_state_transition_for_stopped_workers.yaml b/docs/source/changes/234.fixed_state_transition_for_stopped_workers.yaml new file mode 100644 index 00000000..30b202e2 --- /dev/null +++ b/docs/source/changes/234.fixed_state_transition_for_stopped_workers.yaml @@ -0,0 +1,11 @@ +category: fixed +summary: "Fixed state transition for stopped workers" +description: | + Fixes an unexpected behaviour for Drones in AvailableState in case the + HTCondor daemon on nodes is shutdown automatically causing the machine status + to be NotAvailable, while the resource status continues to be Running. In hat + case the drone state is re-set to IntegratingState. Since HTCondor is not + restarted, the Drone remains in this state forever. +pull requests: +- 234 +version: 0.7.0 diff --git a/docs/source/changes/241.fixed_recurrent_cancelation_of_timeouted_jobs.yaml b/docs/source/changes/241.fixed_recurrent_cancelation_of_timeouted_jobs.yaml new file mode 100644 index 00000000..7b39a67b --- /dev/null +++ b/docs/source/changes/241.fixed_recurrent_cancelation_of_timeouted_jobs.yaml @@ -0,0 +1,12 @@ +category: fixed +summary: "Fixing recurrent cancellation of jobs TIMEOUTED in Slurm" +description: | + Fixed a problem where Slurm jobs in status TIMEOUT are not handled correctly. + Slurm TIMEOUT state were handled as `ResourceStatus.Error` causing TARDIS to + repeatedly cleanup the job from the batch system using `scancel`. Now timeouted + drones in Slurm are handled as `ResourceStatus.Deleted` instead. +issues: +- 240 +pull requests: +- 241 +version: 0.7.0 diff --git a/docs/source/changes/244.add_arguments_htcondor_jdl_parameter.yaml b/docs/source/changes/244.add_arguments_htcondor_jdl_parameter.yaml index 37876b27..8f74cb80 100644 --- a/docs/source/changes/244.add_arguments_htcondor_jdl_parameter.yaml +++ b/docs/source/changes/244.add_arguments_htcondor_jdl_parameter.yaml @@ -1,9 +1,10 @@ category: added summary: "Add support for passing environment variables as executable arguments to support HTCondor grid universe" description: | - In order to properly identify started drones in the overlay batch system and to limit the amount of resources - (CPU cores, memory, disk) announced to be available, a set of environment variables needs to be set inside the drone. - In case of the HTCondor grid universe such an environment is usually dropped by the Grid Compute Element. Therefore - the possibility to pass the environment variables using executable arguments has been added. -pull_requests: - - 224 + In order to properly identify started drones in the overlay batch system + and to limit the amount of resources (CPU cores, memory, disk) announced to + be available, a set of environment variables needs to be set inside the drone. + In case of the HTCondor grid universe such an environment is usually dropped + by the Grid Compute Element. Therefore the possibility to pass the environment + variables using executable arguments has been added. +version: 0.7.0 diff --git a/docs/source/changes/247.change_drone_state_initialisation.yaml b/docs/source/changes/247.change_drone_state_initialisation.yaml index e604dc52..2f8fb070 100644 --- a/docs/source/changes/247.change_drone_state_initialisation.yaml +++ b/docs/source/changes/247.change_drone_state_initialisation.yaml @@ -1,10 +1,12 @@ category: changed summary: "Change drone state initialisation and notification of plugins" description: | - The initialisation procedure and the notification of the plugins is changed to fix a bug occurring on restarts of - Drones. A newly created Drone is now initialised with ``state = None`` and all plugins are notified first state - change ``None`` -> ``RequestState``. The Drone is now inserted in the `SqliteRegistry` when it state changes to - ``RequestState`` and all subsequent changes are DB updates. So, failing duplicated inserts due to the unique - requirement of the ``drone_uuid`` are prevented in case a Drone changes back to ``BootingState`` again. -pull_requests: - - 247 + The initialisation procedure and the notification of the plugins is + changed to fix a bug occurring on restarts of Drones. A newly created Drone + is now initialised with ``state = None`` and all plugins are notified first + state change ``None`` -> ``RequestState``. The Drone is now inserted in the + `SqliteRegistry` when it state changes to ``RequestState`` and all + subsequent changes are DB updates. So, failing duplicated inserts due to the + unique requirement of the ``drone_uuid`` are prevented in case a Drone + changes back to ``BootingState`` again. +version: 0.7.0 diff --git a/docs/source/changes/249_fix_sqlite_registry_remote_uuid_update.yaml b/docs/source/changes/249_fix_sqlite_registry_remote_uuid_update.yaml index 33627cb8..89e3dda7 100644 --- a/docs/source/changes/249_fix_sqlite_registry_remote_uuid_update.yaml +++ b/docs/source/changes/249_fix_sqlite_registry_remote_uuid_update.yaml @@ -1,10 +1,10 @@ category: fixed summary: "Update the remote_resource_uuid in sqlite registry on a each update" description: | - The change drone state initialisation update revealed a bug in TARDIS. The ``remote_resource_uuid`` in the - ``SqliteRegistry`` plugin is not updated at all. As a result, TARDIS keeps crashing on restarts due to the missing - ``remote_resource_uuid`` until the DB has been removed. -pull_requests: - - 249 + The change drone state initialisation update revealed a bug in TARDIS. + The ``remote_resource_uuid`` in the ``SqliteRegistry`` plugin is not + updated at all. As a result, TARDIS keeps crashing on restarts due to the + missing ``remote_resource_uuid`` until the DB has been removed. issues: - - 248 +- 248 +version: 0.7.0 diff --git a/docs/source/changes/250.change_rest_api_auth.yaml b/docs/source/changes/250.change_rest_api_auth.yaml new file mode 100644 index 00000000..a60ab6d4 --- /dev/null +++ b/docs/source/changes/250.change_rest_api_auth.yaml @@ -0,0 +1,8 @@ +category: changed +summary: "REST API cookie authentication and refactoring" +description: | + The authentication method was changed to cookie authentication with jwt + tokens. The login router was completly changed and moved to /user. +pull requests: +- 250 +version: 0.7.0 diff --git a/docs/source/changes/252.change_min_core_limit_from_pool_factory.yaml b/docs/source/changes/252.change_min_core_limit_from_pool_factory.yaml index 35062ded..b58c6169 100644 --- a/docs/source/changes/252.change_min_core_limit_from_pool_factory.yaml +++ b/docs/source/changes/252.change_min_core_limit_from_pool_factory.yaml @@ -5,5 +5,4 @@ description: | requested. This overwrites all Standardisers using the minimum parameter in the COBalD pipeline. It turns out that the ``Standardiser`` is not needed anymore, since the ``utilisation`` and ``allocation`` is always 1.0 when no drone is running, so that automatically one is requested. -pull_requests: - - 252 +version: 0.7.0 diff --git a/docs/source/changes/259.fix_rest_api_keyboard_interupt.yaml b/docs/source/changes/259.fix_rest_api_keyboard_interupt.yaml new file mode 100644 index 00000000..c2627cb1 --- /dev/null +++ b/docs/source/changes/259.fix_rest_api_keyboard_interupt.yaml @@ -0,0 +1,9 @@ +category: fixed +summary: "REST API does not suppress KeyboardInterrupt" +description: | + Implements a workaround for a bug in uvicorn that suppresses + KeyboardInterrupt. It allows clean shutdown in both the current cobald + release and master version. +pull requests: +- 259 +version: 0.7.0 diff --git a/docs/source/changes/260.add_remote_drone_draining.yaml b/docs/source/changes/260.add_remote_drone_draining.yaml index b7d9312f..45b95cc2 100644 --- a/docs/source/changes/260.add_remote_drone_draining.yaml +++ b/docs/source/changes/260.add_remote_drone_draining.yaml @@ -1,8 +1,10 @@ category: added summary: "Added support for manual draining of drones using the REST API" description: | - Added limited support to synchronize the state stored in the ``SqliteRegistry`` with the current state of the drone. - Only implemented for drones in ``AvailableState`` which can transition to ``DrainState`` via a remote update of the - ``SqliteRegistry``, i.e. using the REST API. + Added limited support to synchronize the state stored in the ``SqliteRegistry`` + with the current state of the drone. Only implemented for drones in ``AvailableState`` + which can transition to ``DrainState`` via a remote update of the ``SqliteRegistry``, + i.e. using the REST API. pull requests: - 260 +version: 0.7.0 diff --git a/docs/source/changes/263.add_auditor_accounting_plugin,yaml b/docs/source/changes/263.add_auditor_accounting_plugin,yaml new file mode 100644 index 00000000..ecff0b5d --- /dev/null +++ b/docs/source/changes/263.add_auditor_accounting_plugin,yaml @@ -0,0 +1,7 @@ +category: added +summary: "Added Auditor accounting plugin" +description: | + Added Auditor (Accounting Data Handling Toolbox For Opportunistic Resources) + plugin. +pull requests: +- 263 diff --git a/docs/source/changes/267.add_lancium_site_adapter.yaml b/docs/source/changes/267.add_lancium_site_adapter.yaml index 90c4dd70..a314e1a1 100644 --- a/docs/source/changes/267.add_lancium_site_adapter.yaml +++ b/docs/source/changes/267.add_lancium_site_adapter.yaml @@ -4,3 +4,4 @@ description: | A new Lancium compute site adapter has been added to `TARDIS` to use resources provided by the Lancium compute cluster. pull requests: - 267 +version: 0.7.0 diff --git a/docs/source/changes/274.add_openstack_application_cred_support.yaml b/docs/source/changes/274.add_openstack_application_cred_support.yaml index 94861a71..1915ec5c 100644 --- a/docs/source/changes/274.add_openstack_application_cred_support.yaml +++ b/docs/source/changes/274.add_openstack_application_cred_support.yaml @@ -1,9 +1,11 @@ category: added summary: "Added support for application credentials of the OpenStack site adapter" description: | - Newer versions of OpenStack support the creation of application credentials for specific projects. So, the pair of - application_credential_id and application_credential_secret is only valid of a specific project. The OpenStack site - adapter now fully supports the utilization of application credentials to authenticate against the OpenStack API - endpoint. + Newer versions of OpenStack support the creation of application credentials + for specific projects. So, the pair of application_credential_id and + application_credential_secret is only valid of a specific project. The + OpenStack site adapter now fully supports the utilization of application + credentials to authenticate against the OpenStack API endpoint. pull requests: - 274 +version: 0.7.0 diff --git a/docs/source/changes/77.telegraf_config_update.yaml b/docs/source/changes/77.telegraf_config_update.yaml index 501e3017..e8643ffc 100644 --- a/docs/source/changes/77.telegraf_config_update.yaml +++ b/docs/source/changes/77.telegraf_config_update.yaml @@ -1,6 +1,5 @@ category: added -summary: "The machine name has been added as a default tag in the telegraf monitoring\ - \ plugin, can be overwritten." +summary: "The machine name has been added as a default tag in the telegraf monitoring plugin, can be overwritten." description: | The machine name has been added as a default tag in the telegraf monitoring plugin. Can be overwritten by adding `tardis_machine_name` as default tag in the plugin configuration. diff --git a/docs/source/changes/78.improve_logging_of_htcondor_batch_adapter.yaml b/docs/source/changes/78.improve_logging_of_htcondor_batch_adapter.yaml index fc1653c1..996dc703 100644 --- a/docs/source/changes/78.improve_logging_of_htcondor_batch_adapter.yaml +++ b/docs/source/changes/78.improve_logging_of_htcondor_batch_adapter.yaml @@ -1,6 +1,5 @@ category: fixed -summary: "Improved logging of the `HTCondor` batch system adapter and the status changes\ - \ of the drones" +summary: "Improved logging of the `HTCondor` batch system adapter and the status changes of the drones" description: | Drone resource attributes to have been addedd to the debug logging of drone status changes. The `condor_status` command has been added to the debug logging of the htcondor batch system adapter. diff --git a/docs/source/changes/versions.yaml b/docs/source/changes/versions.yaml index 3a9e9c69..5bbe360b 100644 --- a/docs/source/changes/versions.yaml +++ b/docs/source/changes/versions.yaml @@ -1,3 +1,5 @@ +- semver: 0.7.0 + date: '2023-02-24' - semver: 0.6.0 date: '2021-08-09' - semver: 0.5.0 diff --git a/docs/source/services/services.rst b/docs/source/services/services.rst index 6e2e6551..78056505 100644 --- a/docs/source/services/services.rst +++ b/docs/source/services/services.rst @@ -85,10 +85,6 @@ Available configuration options +----------------+-------------------------------------------------------------------+-----------------+ | port | Port the REST Service is listening on | **Required** | +----------------+-------------------------------------------------------------------+-----------------+ - | secret_key | 32 byte secret key to generate and validate JWTs | **Required** | - +----------------+-------------------------------------------------------------------+-----------------+ - | algorithm | Algorithm to generate and validate JWTs (default: HS256) | **Optional** | - +----------------+-------------------------------------------------------------------+-----------------+ | users | List of user entries allowed to use the REST service. (see below) | **Required** | +----------------+-------------------------------------------------------------------+-----------------+ @@ -104,11 +100,6 @@ Available configuration options | scopes | List of scopes the user is allowed to request. | **Required** | +-----------------+-------------------------------------------------+-----------------+ - .. note:: - - The REST service is using a 32 byte ``secret_key`` to generate and verify JWTs. It can be created using - ``openssl rand -hex 32``. - .. content-tabs:: right-col .. rubric:: Example configuration @@ -120,8 +111,6 @@ Available configuration options !TardisRestApi host: 127.0.0.1 port: 1234 - secret_key: 752e003f636f402cc23728e185ce8c9eef27b7e02cf509b3015f7757e625b8e4 - algorithm: HS256 users: - user_name: tardis hashed_password: $2b$12$c9SSllh1U6tOhIo37sDWF.kdRIU5RQAAOHL9bVYMs2.HluyFE43Uq @@ -182,53 +171,6 @@ Available logging configuration options Convenience Tools ~~~~~~~~~~~~~~~~~ -Generate Token -"""""""""""""" - -.. content-tabs:: left-col - - The ``generate_token`` command provides the possibility to create non-expiring tokens to be used to authenticate - against the REST service (:ref:`see above`). This is useful for automated scripts polling the REST - service. - - .. note:: - - For security reasons it is recommended to use non-expiring tokens only together with ``resources:get`` scope. - - .. table:: Options available to the `generate_token` command - - +-----------------+-------------------------------------------------------------------------------------------+-----------------+ - | Option | Short Description | Requirement | - +=================+===========================================================================================+=================+ - | user_name | The user name the token includes | **Required** | - +-----------------+-------------------------------------------------------------------------------------------+-----------------+ - | scopes | List of scopes the token should have. Comma separated list. | **Required** | - +-----------------+-------------------------------------------------------------------------------------------+-----------------+ - | config-file | The ``COBalD/TARDIS`` configuration file containing the configuration of the REST service | **Optional** | - +-----------------+-------------------------------------------------------------------------------------------+-----------------+ - | secret-key | The secret key to generate the token | **Optional** | - +-----------------+-------------------------------------------------------------------------------------------+-----------------+ - | algorithm | The algorithm to generate the token. For example ``HS256`` | **Optional** | - +-----------------+-------------------------------------------------------------------------------------------+-----------------+ - - Either ``config-file`` or ``secret-key`` and ``algorithm`` needs to be specified. - -.. content-tabs:: right-col - - .. rubric:: Examples - - .. code-block:: shell - - generate_token --user-name tardis --scopes resources:get --config-file tardis.yml - #alternatively - python -m tardis.rest.token_generator --user-name tardis --scopes resources:get --config-file tardis.yml - - .. code-block:: shell - - generate_token --user-name tardis --scopes resources:get --secret-key ... --algorithm HS256 - #alternatively - python -m tardis.rest.token_generator --user-name tardis --scopes resources:get --secret-key ... --algorithm HS256 - Hash Credentials """""""""""""""" diff --git a/tardis/__about__.py b/tardis/__about__.py index 795fd3fb..63470bc7 100644 --- a/tardis/__about__.py +++ b/tardis/__about__.py @@ -3,8 +3,8 @@ __summary__ = "Transparent Adaptive Resource Dynamic Integration System" __url__ = "https://github.com/matterminers/tardis" -__version__ = "0.6.0" +__version__ = "0.7.0" __author__ = "Manuel Giffels, Matthias Schnepf" __email__ = "giffels@gmail.com" -__copyright__ = "2018 - 2021 %s and Contributors" % __author__ +__copyright__ = "2018 - 2023 %s and Contributors" % __author__ __keywords__ = "asyncio tardis cloud scheduler"