Merge pull request #353 from Ouranosinc/permissions-scheme

Modified permissions scheme

.pylintrc

@@ -130,7 +130,7 @@ score=yes
 [REFACTORING]
 
 # Maximum number of nested blocks for function / method body
-max-nested-blocks=5
+max-nested-blocks=10
 
 # Complete name of functions that never returns. When checking for
 # inconsistent-return-statements if a never returning function is called then
@@ -374,7 +374,7 @@ function-naming-style=snake_case
 #function-rgx=
 
 # Good variable names which should always be accepted, separated by a comma.
-good-names=i,j,k,v,ex,x,y,z,f,fd,h,db,kw,ns,qs,_
+good-names=i,j,k,v,kv,ex,x,y,z,f,fd,h,db,kw,ns,qs,_
 
 # Include a hint for the correct naming format with invalid-name.
 include-naming-hint=no

CHANGES.rst

@@ -9,6 +9,72 @@ Changes
 
 * Nothing yet.
 
+`3.0.0 <https://github.com/Ouranosinc/Magpie/tree/3.0.0>`_ (2020-10-19)
+------------------------------------------------------------------------------------
+
+Features / Changes
+~~~~~~~~~~~~~~~~~~~~~
+* Adjust ``alembic`` migration scripts to employ date-ordered naming convention to help searching features within them.
+* Add ``DENY`` permission access concept with new ``PermissionSet`` object and ``Access`` enum
+  (resolves `#235 <https://github.com/Ouranosinc/Magpie/issues/235>`_).
+* Remove ``-match`` suffixed entries from ``Permission`` enum in favor of new ``Scope`` enum employed by
+  new ``PermissionSet`` definition.
+* Update permission entries to employ explicit string representation as ``[name]-[access]-[scope]`` in the database
+  (resolves `#342 <https://github.com/Ouranosinc/Magpie/issues/342>`_).
+* Add ``PermissionType`` enum that details the type of permission being represented in any given response
+  (values correspond to types detailed in documentation).
+* Provide new ``permissions`` list in applicable API responses, with explicit ``name``, ``access``, ``scope`` and
+  ``type`` fields for each ``PermissionSet`` represented as individual JSON object. Responses will also return the
+  *explicit* string representations (see above) combined with the older *implicit* representation still returned
+  in ``permission_names`` field for backward compatibility
+  (note: ``DENY`` elements are only represented as *explicit* as there was no such *implicit* permissions before).
+* Add more documentation details and examples about new permission concepts introduced.
+* Add ``DELETE`` request views with ``permission`` object provided in body to allow deletion using ``PermissionSet``
+  JSON representation instead of literal string by path variable.
+  Still support ``permission_name`` path variable requests for backward compatibility for equivalent names.
+* Add ``POST`` request support of ``permission`` JSON representation of ``PermissionSet`` provided in request body.
+  Fallback to ``permission_name`` field for backward compatibility if equivalent ``permission`` is not found.
+* Add new ``PUT`` request that updates a *possibly* existing ``permission`` (or create it if missing) without needing
+  to execute any prior ``GET`` and/or ``DELETE`` requests that would normally be required to validate the existence or
+  not of previously defined ``permission`` to avoid HTTP Conflict on ``POST``. This allows quicker changes of ``access``
+  and ``scope`` modifiers applied on a given ``permission`` with a single operation
+  (see details in issue `#342 <https://github.com/Ouranosinc/Magpie/issues/342>`_).
+* Add many omitted tests regarding validation of operations on user/group service/resource permissions API routes.
+* Add functional tests that evaluate ``MagpieAdapter`` behaviour and access control of service/resource from resolution
+  of effective permissions upon incoming requests as they would be received by `Twitcher` proxy.
+* Add ``Cache-Control: no-cache`` header support during ACL resolution of effective permissions on service/resource to
+  ignore any caching optimization provided by ``beaker``.
+* Add resource of type ``Process`` for ``ServiceWPS`` which can take advantage of new effective permission resolution
+  method shared across service types to apply ``DescribeProcess`` and ``Execute`` permission on per-``Process`` basis
+  (``match`` scope) or globally for all processes using permission on the parent WPS service (``recursive`` scope).
+  (resolves `#266 <https://github.com/Ouranosinc/Magpie/issues/266>`_).
+* Modify all implementations of ``Service`` to support effective permission resolution to natively support new
+  permissions modifiers ``Access`` and ``Scope``.
+* Adjust all API routes that provide ``effective`` query parameter to return resolved effective permissions of the
+  ``User`` onto the targeted ``Resource``, and this for all applicable permissions on this ``Resource``, using new
+  ``Access`` permission modifier.
+* Adjust UI pages to provide selector of ``Access`` and ``Scope`` modifiers for all available permission names.
+* Change UI permission pages to *Apply* batch edition of multiple entries simultaneously instead of one at the time.
+* Improve rendering of UI disabled items such as inactive checkboxes or selectors when not applicable for given context.
+* Refactor UI tree renderer to reuse same code for both ``User`` and ``Group`` resource permissions.
+* Add UI button on ``User`` edit page to test its *effective permission* on a given resource.
+  Must be in *inherited permissions* display mode to have access to test button, in order to help understand the result.
+
+* | Upgrade migration script is added to convert existing implicit names to new explicit permission names.
+  |
+  | **WARNING**:
+  | Downgrade migration drops any ``DENY`` permission that would be added in future versions,
+    as they do not exist prior to this introduced version. The same applies for ``Process`` resources.
+
+Bug Fixes
+~~~~~~~~~~~~~~~~~~~~~
+* Fix incorrect regex employed for validation of service URL during registration.
+* Replace HTTP status code ``400`` by ``403`` and ``422`` where applicable for invalid resource creation due to failing
+  validations against reference parent service (relates to `#359 <https://github.com/Ouranosinc/Magpie/issues/359>`_).
+* Fix UI rendering of ``Push to Phoenix`` notification when viewing service page with type ``WPS``.
+* Fix UI rendering of some incorrect title background color for alert notifications.
+* Fix UI rendering of tree view items with collapsible/expandable resource nodes.
+
 `2.0.1 <https://github.com/Ouranosinc/Magpie/tree/2.0.1>`_ (2020-09-30)
 ------------------------------------------------------------------------------------
 

Makefile

@@ -19,7 +19,7 @@ MAKEFILE_NAME := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))
 # Application
 APP_ROOT    := $(abspath $(lastword $(MAKEFILE_NAME))/..)
 APP_NAME    := magpie
-APP_VERSION ?= 2.0.1
+APP_VERSION ?= 3.0.0
 APP_INI     ?= $(APP_ROOT)/config/$(APP_NAME).ini
 
 # guess OS (Linux, Darwin,...)
@@ -189,18 +189,29 @@ clean-docker: docker-clean	## alias for 'docker-clean' target
 
 ## --- Database targets --- ##
 
+.PHONY: _alembic
+_alembic: conda-env
+	@bash -c '$(CONDA_CMD) test -f "$(CONDA_ENV_PATH)/bin/alembic" || pip install $(PIP_XARGS) alembic'
+
 .PHONY: migrate
 migrate: database-migration		## alias to 'database-migration'
 
 .PHONY: database-migration
-database-migration: conda-env	## run postgres database migration with alembic
-	@bash -c '$(CONDA_CMD) test -f "$(CONDA_ENV_PATH)/bin/alembic" || "$(MAKE)" -C install'
+database-migration: conda-env _alembic 	## run postgres database migration with alembic
 	@echo "Running database migration..."
 	@bash -c '$(CONDA_CMD) alembic -c "$(APP_INI)" upgrade head'
 
+.PHONY: database-history
+database-history: conda-env _alembic    ## obtain database revision history
+	@bash -c '$(CONDA_CMD) alembic -c "$(APP_INI)" history'
+
 .PHONY: database-revision
-database-revision: conda-env	## retrieve current database revision
-	@bash -c '$(CONDA_CMD) test -f "$(CONDA_ENV_PATH)/bin/alembic" || "$(MAKE)" -C install'
+database-revision: conda-env _alembic   ## create a new database revision
+	@[ "${DOC}" ] || ( echo ">> 'DOC' is not set. Provide a description."; exit 1 )
+	@bash -c '$(CONDA_CMD) alembic -c "$(APP_INI)" revision -m $(DOC)'
+
+.PHONY: database-version
+database-version: conda-env _alembic 	## retrieve current database revision ID
 	@echo "Fetching database revision..."
 	@bash -c '$(CONDA_CMD) alembic -c "$(APP_INI)" current'
 

README.rst

@@ -29,13 +29,13 @@ Behind the scene, it uses `Ziggurat-Foundations`_ and `Authomatic`_.
     :alt: Requires Python 2.7, 3.5+
     :target: https://www.python.org/getit
 
-.. |commits-since| image:: https://img.shields.io/github/commits-since/Ouranosinc/Magpie/2.0.1.svg
+.. |commits-since| image:: https://img.shields.io/github/commits-since/Ouranosinc/Magpie/3.0.0.svg
     :alt: Commits since latest release
-    :target: https://github.com/Ouranosinc/Magpie/compare/2.0.1...master
+    :target: https://github.com/Ouranosinc/Magpie/compare/3.0.0...master
 
-.. |version| image:: https://img.shields.io/badge/tag-2.0.1-blue.svg?style=flat
+.. |version| image:: https://img.shields.io/badge/tag-3.0.0-blue.svg?style=flat
     :alt: Latest Tag
-    :target: https://github.com/Ouranosinc/Magpie/tree/2.0.1
+    :target: https://github.com/Ouranosinc/Magpie/tree/3.0.0
 
 .. |dependencies| image:: https://pyup.io/repos/github/Ouranosinc/Magpie/shield.svg
     :alt: Dependencies Status
@@ -45,9 +45,9 @@ Behind the scene, it uses `Ziggurat-Foundations`_ and `Authomatic`_.
     :alt: Travis-CI Build Status (master branch)
     :target: https://travis-ci.com/Ouranosinc/Magpie
 
-.. |travis_tagged| image:: https://img.shields.io/travis/com/Ouranosinc/Magpie/2.0.1.svg?label=2.0.1
+.. |travis_tagged| image:: https://img.shields.io/travis/com/Ouranosinc/Magpie/3.0.0.svg?label=3.0.0
     :alt: Travis-CI Build Status (latest tag)
-    :target: https://github.com/Ouranosinc/Magpie/tree/2.0.1
+    :target: https://github.com/Ouranosinc/Magpie/tree/3.0.0
 
 .. |readthedocs| image:: https://img.shields.io/readthedocs/pavics-magpie
     :alt: Readthedocs Build Status (master branch)
@@ -118,8 +118,8 @@ Following most recent variants are available:
     * - Magpie
       - Twitcher |br|
         (with integrated ``MagpieAdapter``)
-    * - pavics/magpie:2.0.1
-      - pavics/twitcher:magpie-2.0.1
+    * - pavics/magpie:3.0.0
+      - pavics/twitcher:magpie-3.0.0
     * - pavics/magpie:latest
       - pavics/twitcher:magpie-latest
 

config/magpie.ini

@@ -76,6 +76,7 @@ threads=4
 [alembic]
 script_location = %(here)s/../magpie/alembic
 #sqlalchemy.url = postgresql://${POSTGRES_USERNAME}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}/${POSTGRES_DB}
+file_template = %%(year)d-%%(month).2d-%%(day).2d_%%(rev)s_%%(slug)s
 
 ###
 # logging configuration

config/permissions.cfg

@@ -9,13 +9,25 @@
 #     to process all '.cfg' files found under it one-by-one as separate 'permissions' configurations.
 #
 # Parameters:
+# -----------
 #   service:              service name to receive the permission (directly on it if no 'resource' mentioned, must exist)
 #   resource (optional):  tree path of the service's resource (ex: /res1/sub-res2/sub-sub-res3)
 #   user and/or group:    user/group for which to apply the permission (create if missing, see below)
-#   permission:           name of the permission to be applied (see 'magpie/permissions.py' for supported values)
+#   permission:           name or object of the permission to be applied (see 'magpie.permissions' for supported values)
 #   action:               one of [create, remove] (default: create)
 #
+# Permission:
+# -----------
+#   When provided as string name, it is better to provide the explicit format "[name]-[access]-[scope]" to ensure
+#   correct interpretation, although implicit permission string is supported. Object definition is also possible:
+#
+#   permission:
+#     name:               name of the permission as allowed for the service/resource (e.g.: read, write, etc.)
+#     access:             access rule for the permission (e.g.: allow/deny)
+#     scope:              scope of permission, for tree inheritance or explicitly for resource (e.g.: recursive/match)
+#
 # Default behaviour:
+# ------------------
 #   - create missing resources if supported by the service (and tree automatically resolvable), then apply permissions.
 #   - create missing user/group if required (default user created: (group: anonymous, password: 12345).
 #   - applicable service, user or group is missing, corresponding permissions are ignored and not updated.

docs/configuration.rst

@@ -342,7 +342,7 @@ remain available as described at the start of the `Configuration`_ section.
     This value **MUST** be defined before starting the application in order to move on to user accounts and permissions
     creation in your `Magpie` instance. The application will quit with an error if this value cannot be found.
 
-  .. versionchanged:: 2.0.0
+  .. versionchanged:: 2.0
     Prior to this version, a default value was employed if this setting not provided. Later `Magpie` version now
     require an explicit definition of this parameter to avoid weak default configuration making the protected system
     prone to easier breaches. This also avoids incorrect initial setup of special :term:`User`s with that temporary
@@ -379,7 +379,7 @@ remain available as described at the start of the `Configuration`_ section.
   could cause other operations to fail drastically since this special user will be employed by other `Magpie` internal
   operations such as :ref:`Service Synchronization` and setup during the application startup.
 
-  .. versionchanged:: 2.0.0
+  .. versionchanged:: 2.0
     Prior to this version, a default value was employed if this setting was not provided. Later `Magpie` version now
     require an explicit definition of this parameter to avoid weak default configuration making the protected system
     prone to easier breaches. This value **MUST** be defined before starting the application in order to resume to any
@@ -391,7 +391,7 @@ remain available as described at the start of the `Configuration`_ section.
 
   Password of the default 'administrator' :term:`User` generated by the application (see ``MAGPIE_ADMIN_USER`` details).
 
-  .. versionchanged:: 2.0.0
+  .. versionchanged:: 2.0
     Prior to this version, a default value was employed if this setting was not provided. Later `Magpie` version now
     require an explicit definition of this parameter to avoid weak default configuration making the protected system
     prone to easier breaches. This value **MUST** be defined before starting the application in order to resume to any
@@ -424,7 +424,7 @@ remain available as described at the start of the `Configuration`_ section.
 - | ``MAGPIE_LOGGED_PERMISSION`` [constant]
   | (Value: ``"MAGPIE_LOGGED_USER"``)
 
-  .. versionadded:: 2.0.0
+  .. versionadded:: 2.0
 
   Defines a special condition of :term:`Access Permissions` related to the :term:`Logged User` session and the
   targeted :term:`User` by the request. See details in :ref:`Route Access` for when it applies.
@@ -444,7 +444,7 @@ remain available as described at the start of the `Configuration`_ section.
     but this same user will receive a forbidden response if using is ID in the path if he doesn't have required
     privileges.
 
-  .. versionchanged:: 2.0.0
+  .. versionchanged:: 2.0
     Even without administrative access rights, the :term:`Logged User` is allowed to obtain some additional details
     about the targeted :term:`User` of the request path if it corresponds to itself. See ``MAGPIE_LOGGED_PERMISSION``
     and :ref:`Route Access` for further details.
@@ -474,7 +474,7 @@ remain available as described at the start of the `Configuration`_ section.
   This parameter is enforced to be equal to ``MAGPIE_ANONYMOUS_USER``. It is preserved for backward compatibility of
   migration scripts and external libraries that specifically refer to this parameter.
 
-  .. versionchanged::
+  .. versionchanged:: 2.0
     The :term:`Group` generated by this configuration cannot be modified to remove :term:`User` memberships or change
     other metadata associated to it.
 
@@ -493,7 +493,7 @@ remain available as described at the start of the `Configuration`_ section.
 
   Name of a generic :term:`Group` created to associate registered :term:`User` memberships in the application.
 
-  .. versionchanged:: 2.0.0
+  .. versionchanged:: 2.0
     New :term:`User` are **NOT** automatically added to this :term:`Group` anymore. This :term:`Group` remains
     available for testing and backward compatibility reasons, but doesn't have any special connotation and can be
     modified just as any other normal :term:`Group`.
@@ -518,7 +518,7 @@ remain available as described at the start of the `Configuration`_ section.
 - | ``MAGPIE_PASSWORD_MIN_LENGTH``
   | (Default: ``12``)
 
-  .. versionadded:: 2.0.0
+  .. versionadded:: 2.0
     Minimum length of the password for :term:`User` creation or update.
 
   .. note::
@@ -594,7 +594,7 @@ Following settings define parameters required by `Twitcher`_ (OWS Security Proxy
 - | ``TWITCHER_HOST``
   | (Default: None)
 
-  .. versionadded:: 2.0.0
+  .. versionadded:: 2.0
 
   Specifies the explicit hostname to employ in combination with ``TWITCHER_PROTECTED_PATH`` to form the complete base
   service protected URL. Ignored if ``TWITCHER_PROTECTED_URL`` was provided directly.
@@ -660,7 +660,7 @@ configuration names are supported where mentioned.
 
   Database connection username to retrieve `Magpie` data stored in `PostgreSQL`_.
 
-  .. versionchanged:: 1.9.0
+  .. versionchanged:: 1.9
       On top of ``MAGPIE_POSTGRES_USERNAME``, environment variable ``POSTGRES_USERNAME`` and setting
       ``postgres.username`` are all supported interchangeably. For backward compatibility, all above variants with
       ``user`` instead of ``username`` (with corresponding lower/upper case) are also verified for potential
@@ -672,7 +672,7 @@ configuration names are supported where mentioned.
 
   Database connection password to retrieve `Magpie` data stored in `PostgreSQL`_.
 
-  .. versionchanged:: 1.9.0
+  .. versionchanged:: 1.9
     Environment variable ``POSTGRES_PASSWORD`` and setting ``postgres.password`` are also supported if not previously
     identified by their `Magpie`-prefixed variants.
 
@@ -681,7 +681,7 @@ configuration names are supported where mentioned.
 
   Database connection host location to retrieve `Magpie` data stored in `PostgreSQL`_.
 
-  .. versionchanged:: 1.9.0
+  .. versionchanged:: 1.9
     Environment variable ``POSTGRES_HOST`` and setting ``postgres.host`` are also supported if not previously
     identified by their `Magpie`-prefixed variants.
 
@@ -690,7 +690,7 @@ configuration names are supported where mentioned.
 
   Database connection port to retrieve `Magpie` data stored in `PostgreSQL`_.
 
-  .. versionchanged:: 1.9.0
+  .. versionchanged:: 1.9
     Environment variable ``POSTGRES_PORT`` and setting ``postgres.port`` are also supported if not previously
     identified by their `Magpie`-prefixed variants.
 
@@ -699,7 +699,7 @@ configuration names are supported where mentioned.
 
   Name of the database located at the specified connection to retrieve `Magpie` data stored in `PostgreSQL`_.
 
-  .. versionchanged:: 1.9.0
+  .. versionchanged:: 1.9
     Environment variable ``POSTGRES_DB`` and setting ``postgres.db``, as well as the same variants with ``database``
     instead of ``db``, are also supported if not previously identified by their `Magpie`-prefixed variants.