Implement combined Geoserver service and Workspace/Layer effective permissions

CHANGES.rst

@@ -12,6 +12,51 @@ Changes
 `3.20.0 <https://github.com/Ouranosinc/Magpie/tree/3.20.0>`_ (2022-01-06)
 ------------------------------------------------------------------------------------
 
+Features / Changes
+~~~~~~~~~~~~~~~~~~~~~
+* Add missing ``ServiceWFS`` permissions according to `OGC WFS standard <https://www.ogc.org/standards/wfs>`_.
+* Add missing ``DescribeLayer`` permission to ``ServiceGeoserverWMS`` according
+  to `GeoServer WMS implementation <https://docs.geoserver.org/latest/en/user/services/wms/reference.html>`_.
+* Add support of specific hierarchy of ``Resource`` type ``Layer`` nested under ``Workspace``
+  for ``ServiceGeoserverWMS``.
+* Add support of ``Resource`` type ``Layer`` under ``ServiceWFS``.
+* Allow ``Resource`` and ``Service`` name to contain colon (``:``) character in order to define scoped names
+  as it is often the case for ``Layer`` names.
+* Add ``child_structure_allowed`` attribute to ``Service`` implementations allowing them to define specific path-like
+  structures of allowed ``Resource`` types hierarchies in order to control at which level and which combinations
+  of nested ``Resource`` types are valid under their root ``Service``. When not defined under a ``Service``
+  implementation, any defined ``Resource`` type will remain available for creation at any level of the hierarchy,
+  unless the corresponding ``Resource`` in the tree already defined ``child_resource_allowed = False``. This was
+  already the original behaviour in previous versions.
+* Add ``GET /resources/{id}/types`` endpoint that allows retrieval of applicable children ``Resource`` types under
+  a given ``Resource`` considering the nested hierarchy definition of its root ``Service`` defined by the new
+  attribute ``child_structure_allowed``.
+* Add ``child_structure_allowed`` attribute to the response of ``GET /service/{name}`` endpoint.
+  For backward compatibility, ``resource_types_allowed`` parameter already available in the same response will continue
+  to report all possible ``Resource`` types *at any level* under the ``Service`` hierarchy, although not necessarily
+  applicable as immediate child ``Resource`` under that ``Service``.
+* Add ``configurable`` attribute to ``Service`` types that supports custom definitions modifying their behaviour.
+* Add ``service_configurable`` to response of ``GET /service/{name}`` endpoint.
+* Adjust UI to consider ``child_structure_allowed`` definitions to propose only applicable ``Resource`` types in the
+  combobox when creating a new ``Resource`` in the tree hierarchy.
+* Add UI submission field to provide ``Service`` JSON configuration at creation when supported by the type.
+
+Bug Fixes
+~~~~~~~~~~~~~~~~~~~~~
+* Remove invalid ``params_expected`` parameter from ``Service`` implementations (``ServiceAccess``, ``ServiceAPI``,
+  ``ServiceTHREDDS``) that don't make use of it since they don't derive from ``ServiceOWS``.
+* Fix base ``Permission`` definitions for all variants of `WMS` according to their reference implementations.
+* Remove multiple invalid schema path definitions that are not mapped against any concrete API endpoint.
+* Fix reporting of ``Service`` configuration for any type that supports it. Unless overridden during creation with a
+  custom configuration, ``ServiceTHREDDS`` implementation would not report their default configuration and would
+  instead return ``null``, making it difficult to know from the API if default or no configuration was being applied
+  for a given ``Service``.
+* Fix `Effective Resolution` of ``Permission`` applied for ``ServiceGeoserverWMS`` to consider ``Scope`` modifier
+  of ``Service`` and ``Workspace`` for access to be resolved at the ``Layer`` level.
+
+`3.20.0 <https://github.com/Ouranosinc/Magpie/tree/3.20.0>`_ (2022-01-06)
+------------------------------------------------------------------------------------
+
 Features / Changes
 ~~~~~~~~~~~~~~~~~~~~~
 * Add improved UI display of long ``Permission`` titles for ``Resource`` hierarchy tree headers.

docs/services.rst

@@ -78,8 +78,8 @@ On top of the above methods, the following attributes must be defined.
         :Term:`Allowed Permissions`, their type and further nested children :term:`Resource`.
     * - :attr:`ServiceInterface.params_expected` |br| (``List[str]``)
       - Represents specific parameter names that can be preprocessed during HTTP request parsing to ease following
-        resolution of :term:`ACL` use cases. Employed most notably by :term:`Service` implementations based on
-        :class:`magpie.services.ServiceOWS`.
+        resolution of :term:`ACL` use cases. Employed only by :term:`Service` implementations derived from
+        :class:`magpie.services.ServiceOWS`. Can be omitted otherwise.
 
 
 .. _services_available:

magpie/api/exception.py

@@ -44,6 +44,7 @@
 
 # utility parameter validation regexes for 'matches' argument
 PARAM_REGEX = r"^[A-Za-z0-9]+(?:[\s_\-\.][A-Za-z0-9]+)*$"    # request parameters
+SCOPE_REGEX = r"^[A-Za-z0-9]+(?:[\:\s_\-\.][A-Za-z0-9]+)*$"  # allow scoped names (e.g.: 'namespace:value')
 EMAIL_REGEX = colander.EMAIL_RE
 UUID_REGEX = colander.UUID_REGEX
 URL_REGEX = colander.URL_REGEX

magpie/api/management/group/__init__.py

@@ -17,6 +17,5 @@ def includeme(config):
     config.add_route(**s.service_api_route_info(s.GroupResourcesAPI))
     config.add_route(**s.service_api_route_info(s.GroupResourcePermissionsAPI))
     config.add_route(**s.service_api_route_info(s.GroupResourcePermissionAPI))
-    config.add_route(**s.service_api_route_info(s.GroupResourceTypesAPI))
 
     config.scan()

magpie/api/management/resource/__init__.py

@@ -10,5 +10,6 @@ def includeme(config):
     config.add_route(**s.service_api_route_info(s.ResourcesAPI))
     config.add_route(**s.service_api_route_info(s.ResourceAPI))
     config.add_route(**s.service_api_route_info(s.ResourcePermissionsAPI))
+    config.add_route(**s.service_api_route_info(s.ResourceTypesAPI))
 
     config.scan()

magpie/api/management/resource/resource_utils.py

@@ -58,6 +58,7 @@ def check_valid_service_or_resource_permission(permission_name, service_or_resou
 
 
 def check_valid_service_resource(parent_resource, resource_type, db_session):
+    # type: (ServiceOrResourceType, Str, Session) -> models.Service
     """
     Checks if a new Resource can be contained under a parent Resource given the requested type and the corresponding
     Service under which the parent Resource is already assigned.
@@ -69,20 +70,34 @@ def check_valid_service_resource(parent_resource, resource_type, db_session):
     """
     parent_type = parent_resource.resource_type_name
     parent_msg_err = "Child resource not allowed for specified parent resource type '{}'".format(parent_type)
-    ax.verify_param(models.RESOURCE_TYPE_DICT[parent_type].child_resource_allowed, is_equal=True,
-                    param_compare=True, http_error=HTTPForbidden, msg_on_fail=parent_msg_err)
+    ax.verify_param(models.RESOURCE_TYPE_DICT[parent_type].child_resource_allowed, is_true=True,
+                    http_error=HTTPForbidden, msg_on_fail=parent_msg_err)
     root_service = get_resource_root_service(parent_resource, db_session=db_session)
     ax.verify_param(root_service, not_none=True, http_error=HTTPInternalServerError,
                     msg_on_fail="Failed retrieving 'root_service' from db")
     ax.verify_param(root_service.resource_type, is_equal=True, http_error=HTTPInternalServerError,
                     param_name="resource_type", param_compare=models.Service.resource_type_name,
                     msg_on_fail="Invalid 'root_service' retrieved from db is not a service")
-    ax.verify_param(SERVICE_TYPE_DICT[root_service.type].child_resource_allowed, is_equal=True,
-                    param_compare=True, http_error=HTTPForbidden,
+    root_svc_cls = SERVICE_TYPE_DICT[root_service.type]
+    ax.verify_param(root_svc_cls.child_resource_allowed, is_true=True, http_error=HTTPForbidden,
                     msg_on_fail="Child resource not allowed for specified service type '{}'".format(root_service.type))
     ax.verify_param(resource_type, is_in=True, http_error=HTTPForbidden,
-                    param_name="resource_type", param_compare=SERVICE_TYPE_DICT[root_service.type].resource_type_names,
+                    param_name="resource_type", param_compare=root_svc_cls.resource_type_names,
                     msg_on_fail="Invalid 'resource_type' specified for service type '{}'".format(root_service.type))
+    ax.verify_param(
+        root_svc_cls.validate_nested_resource_type(parent_resource, resource_type), is_true=True,
+        param_content={
+            "resource_structure_allowed": root_svc_cls.child_structure_allowed,
+            "resource_types_allowed": [
+                res.resource_type for res in root_svc_cls.nested_resource_allowed(parent_resource)
+            ]
+        },
+        http_error=HTTPUnprocessableEntity,
+        msg_on_fail=(
+            "Invalid 'resource_type' specified for service type '{}' is not allowed at this position "
+            "under '{}' resource.".format(root_service.type, parent_type)
+        )
+    )
     return root_service
 
 
@@ -273,6 +288,7 @@ def get_resource_root_service_impl(resource, request):
 def create_resource(resource_name, resource_display_name, resource_type, parent_id, db_session):
     # type: (Str, Optional[Str], Str, int, Session) -> HTTPException
     ax.verify_param(resource_name, param_name="resource_name", not_none=True, not_empty=True,
+                    matches=True, param_compare=ax.SCOPE_REGEX,
                     http_error=HTTPUnprocessableEntity,
                     msg_on_fail="Invalid 'resource_name' specified for child resource creation.")
     ax.verify_param(resource_type, param_name="resource_type", not_none=True, not_empty=True,

magpie/api/management/resource/resource_views.py

@@ -55,7 +55,7 @@ def create_resource_view(request):
     """
     Register a new resource.
     """
-    resource_name = ar.get_value_multiformat_body_checked(request, "resource_name")
+    resource_name = ar.get_multiformat_body(request, "resource_name")
     resource_display_name = ar.get_multiformat_body(request, "resource_display_name", default=resource_name)
     resource_type = ar.get_value_multiformat_body_checked(request, "resource_type")
     parent_id = ar.get_value_multiformat_body_checked(request, "parent_id", check_type=int)
@@ -127,3 +127,46 @@ def get_resource_permissions_view(request):
                                 content={"resource": rf.format_resource(resource, basic_info=True)})
     return ax.valid_http(http_success=HTTPOk, detail=s.ResourcePermissions_GET_OkResponseSchema.description,
                          content=format_permissions(res_perm, PermissionType.ALLOWED))
+
+
+@s.ResourceTypesAPI.get(schema=s.ResourceTypes_GET_RequestSchema, tags=[s.ResourcesTag],
+                        response_schemas=s.ResourceTypes_GET_responses)
+@view_config(route_name=s.ResourceTypesAPI.name, request_method="GET")
+def get_resource_types_view(request):
+    """
+    List all applicable children resource types under another resource within a service hierarchy.
+    """
+    resource = ar.get_resource_matchdict_checked(request, "resource_id")
+
+    def get_res_types(res):
+        svc_root = ru.get_resource_root_service(res, db_session=request.db)
+        svc_impl = SERVICE_TYPE_DICT[svc_root.type]
+        return svc_impl.nested_resource_allowed(res), svc_root
+
+    def get_res_child_allowed(res):
+        # make sure to obtain the specific resource/service implementation to avoid using the default
+        if res.resource_type_name == models.Service.resource_type_name:
+            res_impl = SERVICE_TYPE_DICT[res.type]
+        else:
+            res_impl = models.RESOURCE_TYPE_DICT[res.resource_type_name]
+        return res_impl.child_resource_allowed
+
+    res_types, svc = ax.evaluate_call(lambda: get_res_types(resource),
+                                      fallback=lambda: request.db.rollback(), http_error=HTTPInternalServerError,
+                                      msg_on_fail="Error occurred while computing applicable children resource types.",
+                                      content={"resource": rf.format_resource(resource, basic_info=True)})
+    child_allowed = ax.evaluate_call(lambda: get_res_child_allowed(resource),
+                                     http_error=HTTPInternalServerError,
+                                     msg_on_fail="Error occurred while computing allowed children resource status.",
+                                     content={"resource": rf.format_resource(resource, basic_info=True)})
+    data = {
+        "resource_name": resource.resource_name,
+        "resource_type": resource.resource_type_name,
+        "children_resource_types": list(sorted(res_type.resource_type_name for res_type in res_types)),
+        "children_resource_allowed": child_allowed,
+        "root_service_id": svc.resource_id,
+        "root_service_name": svc.resource_name,
+        "root_service_type": svc.type,
+    }
+    return ax.valid_http(http_success=HTTPOk, content=data,
+                         detail=s.ResourceTypes_GET_OkResponseSchema.description)

magpie/api/management/service/service_formats.py

@@ -65,6 +65,7 @@ def fmt_svc():
             "service{}name".format(sep): str(service.resource_name),
             "service{}type".format(sep): str(service.type),
             "service{}sync_type".format(sep): svc_sync_type,
+            "service{}configurable".format(sep): SERVICE_TYPE_DICT[service.type].configurable,
             "resource{}id".format(sep): service.resource_id,
         }
         if show_public_url:
@@ -74,13 +75,20 @@ def fmt_svc():
             svc_info["service{}url".format(sep)] = str(service.url)
         if basic_info:
             return svc_info
+        svc_type = SERVICE_TYPE_DICT[service.type]
         if show_configuration:
-            svc_info["configuration"] = service.configuration
-        perms = SERVICE_TYPE_DICT[service.type].permissions if permissions is None else permissions
+            # make sure to generate the default configuration if applicable
+            if svc_type.configurable:
+                svc_config = svc_type(service, request=None).get_config()
+            else:
+                svc_config = None
+            svc_info["configuration"] = svc_config
+        perms = svc_type.permissions if permissions is None else permissions
         svc_info.update(format_permissions(perms, permission_type))
         if show_resources_allowed:
-            svc_info["resource_types_allowed"] = sorted(SERVICE_TYPE_DICT[service.type].resource_type_names)
-            svc_info["resource_child_allowed"] = SERVICE_TYPE_DICT[service.type].child_resource_allowed
+            svc_info["resource_child_allowed"] = svc_type.child_resource_allowed
+            svc_info["resource_types_allowed"] = sorted(svc_type.resource_type_names)
+            svc_info["resource_structure_allowed"] = sorted(svc_type.child_structure_allowed)
         return svc_info
 
     return evaluate_call(

magpie/api/management/service/service_utils.py

@@ -67,7 +67,7 @@ def _add_service_magpie_and_phoenix(svc, svc_push, db):
     ax.verify_param(service_url, matches=True, param_compare=ax.URL_REGEX, param_name="service_url",
                     http_error=HTTPBadRequest, msg_on_fail=s.Services_POST_Params_BadRequestResponseSchema.description)
     ax.verify_param(service_name, not_empty=True, not_none=True, matches=True,
-                    param_name="service_name", param_compare=ax.PARAM_REGEX,
+                    param_name="service_name", param_compare=ax.SCOPE_REGEX,
                     http_error=HTTPBadRequest, msg_on_fail=s.Services_POST_Params_BadRequestResponseSchema.description)
     ax.verify_param(models.Service.by_service_name(service_name, db_session=db_session), is_none=True,
                     param_name="service_name", with_param=False, content={"service_name": str(service_name)},

magpie/api/management/service/service_views.py

@@ -113,7 +113,7 @@ def register_service_view(request):
     Registers a new service.
     """
     # accomplish basic validations here, create_service will do more field-specific checks
-    service_name = ar.get_value_multiformat_body_checked(request, "service_name")
+    service_name = ar.get_value_multiformat_body_checked(request, "service_name", pattern=ax.SCOPE_REGEX)
     service_url = ar.get_value_multiformat_body_checked(request, "service_url", pattern=ax.URL_REGEX)
     service_type = ar.get_value_multiformat_body_checked(request, "service_type")
     service_push = asbool(ar.get_multiformat_body(request, "service_push", default=False))
@@ -166,6 +166,9 @@ def select_update(new_value, old_value):
         ax.verify_param(svc_name, not_in=True, param_compare=all_svc_names, with_param=False,
                         http_error=HTTPConflict, content={"service_name": str(svc_name)},
                         msg_on_fail=s.Service_PATCH_ConflictResponseSchema.description)
+        ax.verify_param(svc_name, not_none=True, not_empty=True, matches=True, param_compare=ax.SCOPE_REGEX,
+                        http_error=HTTPBadRequest,
+                        msg_on_fail=s.Service_PATCH_UnprocessableEntityResponseSchema.description)
 
     def update_service_magpie_and_phoenix(_svc, new_name, new_url, svc_push, db_session):
         _svc.resource_name = new_name

magpie/api/management/user/__init__.py

@@ -21,7 +21,6 @@ def includeme(config):
     config.add_route(**s.service_api_route_info(s.UserServicePermissionAPI, **user_kwargs))
     config.add_route(**s.service_api_route_info(s.UserServiceResourcesAPI, **user_kwargs))
     config.add_route(**s.service_api_route_info(s.UserResourcesAPI, **user_kwargs))
-    config.add_route(**s.service_api_route_info(s.UserResourceTypesAPI, **user_kwargs))
     config.add_route(**s.service_api_route_info(s.UserResourcePermissionsAPI, **user_kwargs))
     config.add_route(**s.service_api_route_info(s.UserResourcePermissionAPI, **user_kwargs))
     # Logged User routes
@@ -33,7 +32,6 @@ def includeme(config):
     config.add_route(**s.service_api_route_info(s.LoggedUserServicePermissionAPI, **user_kwargs))
     config.add_route(**s.service_api_route_info(s.LoggedUserServiceResourcesAPI, **user_kwargs))
     config.add_route(**s.service_api_route_info(s.LoggedUserResourcesAPI, **user_kwargs))
-    config.add_route(**s.service_api_route_info(s.LoggedUserResourceTypesAPI, **user_kwargs))
     config.add_route(**s.service_api_route_info(s.LoggedUserResourcePermissionsAPI, **user_kwargs))
     config.add_route(**s.service_api_route_info(s.LoggedUserResourcePermissionAPI, **user_kwargs))
 

magpie/api/requests.py

@@ -311,7 +311,7 @@ def get_service_matchdict_checked(request, service_name_key="service_name"):
     :raises HTTPForbidden: if the requesting user does not have sufficient permission to execute this request.
     :raises HTTPNotFound: if the specified service name does not correspond to any existing service.
     """
-    service_name = get_value_matchdict_checked(request, service_name_key)
+    service_name = get_value_matchdict_checked(request, service_name_key, pattern=ax.SCOPE_REGEX)
     service = ax.evaluate_call(lambda: models.Service.by_service_name(service_name, db_session=request.db),
                                fallback=lambda: request.db.rollback(), http_error=HTTPForbidden,
                                msg_on_fail=s.Service_MatchDictCheck_ForbiddenResponseSchema.description)