Site Accounts service for API keys (#1506)

changelog/unreleased/mentix-clone-fix.md

@@ -0,0 +1,5 @@
+Bugfix: Cloning of internal mesh data lost some values
+
+This update fixes a bug in Mentix that caused some (non-critical) values to be lost during data cloning that happens internally.
+
+https://github.com/cs3org/reva/pull/1457

changelog/unreleased/siteaccs-svc.md

@@ -0,0 +1,7 @@
+Enhancement: Site Accounts service for API keys
+
+This update adds a new service to Reva that handles site accounts creation and management. Registered sites can be assigned an API key through a simple web interface which is also part of this service. This API key can then be used to identify a user and his/her associated (vendor or partner) site.
+
+Furthermore, Mentix was extended to make use of this new service. This way, all sites now have a stable and unique site ID that not only avoids ID collisions but also introduces a new layer of security (i.e., sites can only be modified or removed using the correct API key). 
+
+https://github.com/cs3org/reva/pull/1506

docs/content/en/docs/config/http/services/mentix/_index.md

@@ -10,6 +10,7 @@ description: >
 Mentix (_**Me**sh E**nti**ty E**x**changer_) is a service to read and write mesh topology data to and from one or more sources (e.g., a GOCDB instance) and export it to various targets like an HTTP endpoint or Prometheus.
 {{% /pageinfo %}}
 
+## General settings
 {{% dir name="prefix" type="string" default="mentix" %}}
 The relative root path of all exposed HTTP endpoints of Mentix.
 {{< highlight toml >}}
@@ -42,11 +43,8 @@ Mentix can import mesh data from various sources and write it to one or more tar
 
 __Supported importers:__
 
-- **webapi**
-Mentix can import mesh data via an HTTP endpoint using the `webapi` importer. Data can be sent to the configured relative endpoint (see [here](webapi)).
-
-- **adminapi**
-  Some aspects of Mentix can be administered through an HTTP endpoint using the `adminapi` importer. Queries can be sent to the configured relative endpoint (see [here](adminapi)).
+- **sitereg**
+Mentix can import new sites via an HTTP endpoint using the `sitereg` importer. Data can be sent to the configured relative endpoint (see [here](sitereg)).
 
 ## Exporters
 Mentix exposes its gathered data by using one or more _exporters_. Such exporters can, for example, write the data to a file in a specific format, or offer the data via an HTTP endpoint.
@@ -65,3 +63,30 @@ Mentix exposes its data via an HTTP endpoint using the `webapi` exporter. Data c
           - files:
              - '/usr/share/prom/sciencemesh_services.json'
   ```
+
+## Site Accounts service
+Mentix uses the Reva site accounts service to query information about site accounts. The following settings must be configured properly:
+
+{{% dir name="url" type="string" default="" %}}
+The URL of the site accounts service.
+{{< highlight toml >}}
+[http.services.mentix.accounts]
+url = "https://example.com/accounts"
+{{< /highlight >}}
+{{% /dir %}}
+
+{{% dir name="user" type="string" default="" %}}
+The user name to use for basic HTTP authentication.
+{{< highlight toml >}}
+[http.services.mentix.accounts]
+user = "hans"
+{{< /highlight >}}
+{{% /dir %}}
+
+{{% dir name="password" type="string" default="" %}}
+The user password to use for basic HTTP authentication.
+{{< highlight toml >}}
+[http.services.mentix.accounts]
+password = "secret"
+{{< /highlight >}}
+{{% /dir %}}

docs/content/en/docs/config/http/services/mentix/sitereg/_index.md

@@ -0,0 +1,37 @@
+---
+title: "sitereg"
+linkTitle: "sitereg"
+weight: 10
+description: >
+    Configuration for site registration service
+---
+
+{{% pageinfo %}}
+The site registration service is used to register new and unregister existing sites.
+{{% /pageinfo %}}
+
+The site registration service is used to register new and unregister existing sites.
+
+{{% dir name="endpoint" type="string" default="/sitereg" %}}
+The endpoint of the service.
+{{< highlight toml >}}
+[http.services.mentix.importers.sitereg]
+endpoint = "/reg"
+{{< /highlight >}}
+{{% /dir %}}
+
+{{% dir name="enabled_connectors" type="[]string" default="" %}}
+A list of all enabled connectors for the importer.
+{{< highlight toml >}}
+[http.services.mentix.importers.sitereg]
+enabled_connectors = ["localfile"]
+{{< /highlight >}}
+{{% /dir %}}
+
+{{% dir name="ignore_sm_sites" type="bool" default="false" %}}
+If set to true, registrations from ScienceMesh sites will be ignored.
+{{< highlight toml >}}
+[http.services.mentix.importers.sitereg]
+ignore_sm_sites = true
+{{< /highlight >}}
+{{% /dir %}}

docs/content/en/docs/config/http/services/mentix/webapi/_index.md

@@ -7,49 +7,9 @@ description: >
 ---
 
 {{% pageinfo %}}
-The WebAPI of Mentix supports both importing and exporting of mesh data via an HTTP endpoint. Both the im- and exporter are configured separately.
+The WebAPI of Mentix supports of mesh data via an HTTP endpoint.
 {{% /pageinfo %}}
 
-## Importer
-
-The WebAPI importer receives a single _plain_ Mentix site through an HTTP `POST` request; service types are currently not supported.
-
-The importer supports two actions that must be passed in the URL:
-```
-https://sciencemesh.example.com/mentix/webapi/?action=<value>
-```
-Currently, the following actions are supported:
-- `register`: Registers a new site
-- `unregister`: Unregisters an existing site
-
-For all actions, the site data must be sent as JSON data. If the call succeeded, status 200 is returned.
-
-{{% dir name="endpoint" type="string" default="/sites" %}}
-The endpoint where the mesh data can be sent to.
-{{< highlight toml >}}
-[http.services.mentix.importers.webapi]
-endpoint = "/data"
-{{< /highlight >}}
-{{% /dir %}}
-
-{{% dir name="is_protected" type="bool" default="false" %}}
-Whether the endpoint requires authentication.
-{{< highlight toml >}}
-[http.services.mentix.importers.webapi]
-is_protected = true
-{{< /highlight >}}
-{{% /dir %}}
-
-{{% dir name="enabled_connectors" type="[]string" default="" %}}
-A list of all enabled connectors for the importer. Must always be provided.
-{{< highlight toml >}}
-[http.services.mentix.importers.webapi]
-enabled_connectors = ["localfile"]
-{{< /highlight >}}
-{{% /dir %}}
-
-## Exporter
-
 The WebAPI exporter exposes the _plain_ Mentix data via an HTTP endpoint.
 
 {{% dir name="endpoint" type="string" default="/sites" %}}

docs/content/en/docs/config/http/services/siteacc/_index.md

@@ -0,0 +1,95 @@
+---
+title: "siteacc"
+linkTitle: "siteacc"
+weight: 10
+description: >
+    Configuration for the Site Accounts service
+---
+
+{{% pageinfo %}}
+The site accounts service is used to store and manage site accounts.
+{{% /pageinfo %}}
+
+## General settings
+{{% dir name="prefix" type="string" default="accounts" %}}
+The relative root path of all exposed HTTP endpoints of the service.
+{{< highlight toml >}}
+[http.services.siteacc]
+prefix = "/siteacc"
+{{< /highlight >}}
+{{% /dir %}}
+
+{{% dir name="notifications_mail" type="string" default="" %}}
+An email address where all notifications are sent to.
+{{< highlight toml >}}
+[http.services.siteacc]
+notifications_mail = "[email protected]"
+{{< /highlight >}}
+{{% /dir %}}
+
+## SMTP settings
+{{% dir name="sender_mail" type="string" default="" %}}
+An email address from which all emails are sent.
+{{< highlight toml >}}
+[http.services.siteacc.smtp]
+sender_mail = "[email protected]"
+{{< /highlight >}}
+{{% /dir %}}
+
+{{% dir name="sender_login" type="string" default="" %}}
+The login name.
+{{< highlight toml >}}
+[http.services.siteacc.smtp]
+sender_login = "hans"
+{{< /highlight >}}
+{{% /dir %}}
+
+{{% dir name="sender_password" type="string" default="" %}}
+The password for the login.
+{{< highlight toml >}}
+[http.services.siteacc.smtp]
+password = "secret"
+{{< /highlight >}}
+{{% /dir %}}
+
+{{% dir name="smtp_server" type="string" default="" %}}
+The SMTP server to use.
+{{< highlight toml >}}
+[http.services.siteacc.smtp]
+smtp_server = "smtp.example.com"
+{{< /highlight >}}
+{{% /dir %}}
+
+{{% dir name="smtp_port" type="int" default="25" %}}
+The SMTP server port to use.
+{{< highlight toml >}}
+[http.services.siteacc.smtp]
+smtp_port = 25
+{{< /highlight >}}
+{{% /dir %}}
+
+{{% dir name="disable_auth" type="bool" default="false" %}}
+Whether to disable authentication.
+{{< highlight toml >}}
+[http.services.siteacc.smtp]
+disable_auth = true
+{{< /highlight >}}
+{{% /dir %}}
+
+## Storage settings
+{{% dir name="driver" type="string" default="file" %}}
+The storage driver to use; currently, only `file` is supported.
+{{< highlight toml >}}
+[http.services.siteacc.storage]
+driver = "file"
+{{< /highlight >}}
+{{% /dir %}}
+
+### Storage settings - File driver
+{{% dir name="file" type="string" default="" %}}
+The file location.
+{{< highlight toml >}}
+[http.services.siteacc.storage.file]
+file = "/var/reva/accounts.json"
+{{< /highlight >}}
+{{% /dir %}}

examples/mentix/mentix.toml

@@ -1,9 +1,5 @@
-[shared]
-jwt_secret = "Ment1x-T0pS3cr3t"
-
 [http]
 address = "0.0.0.0:9600"
-enabled_services = ["mentix"]
 
 [http.services.mentix]
 update_interval = "15m"
@@ -25,16 +21,19 @@ endpoint = "/"
 # If this setting is omitted, all connectors will be used as data sources
 enabled_connectors = ["gocdb"]
 
-# Enable the WebAPI importer
-[http.services.mentix.importers.webapi]
+# Enable the site registration importer
+[http.services.mentix.importers.sitereg]
 # For importers, this is obligatory; the connectors will be used as the target for data updates
 enabled_connectors = ["localfile"]
-
-# Enable the AdminAPI importer
-[http.services.mentix.importers.adminapi]
-enabled_connectors = ["localfile"]
-# Should never allow access w/o prior authorization
-is_protected = true
+# If set to true, ScienceMesh sites will be ignored when they try to register
+ignore_sm_sites = false
+
+# Set up the accounts service used to query information about accounts associated with registered sites
+[http.services.mentix.accounts]
+# Depending on where the service is running, localhost may also be used here
+url = "https://sciencemesh.example.com/iop/accounts"
+user = "username"
+password = "userpass"
 
 # Configure the Prometheus Service Discovery:
 [http.services.mentix.exporters.promsd]

examples/siteacc/siteacc.toml

@@ -0,0 +1,19 @@
+[http]
+address = "0.0.0.0:9600"
+
+[http.services.siteacc]
+# All notification emails are sent to this email
+notifications_mail = "[email protected]"
+
+# Set up the storage driver
+[http.services.siteacc.storage]
+driver = "file"
+[http.services.siteacc.storage.file]
+file = "/var/revad/accounts.json"
+
+# The SMTP server used for sending emails
+[http.services.siteacc.smtp]
+sender_mail = "[email protected]"
+smtp_server = "mail.example.com"
+smtp_port = 25
+disable_auth = true

internal/http/services/loader/loader.go

@@ -31,6 +31,7 @@ import (
 	_ "github.com/cs3org/reva/internal/http/services/owncloud/ocdav"
 	_ "github.com/cs3org/reva/internal/http/services/owncloud/ocs"
 	_ "github.com/cs3org/reva/internal/http/services/prometheus"
+	_ "github.com/cs3org/reva/internal/http/services/siteacc"
 	_ "github.com/cs3org/reva/internal/http/services/sysinfo"
 	_ "github.com/cs3org/reva/internal/http/services/wellknown"
 	// Add your own service here

internal/http/services/mentix/mentix.go

@@ -137,12 +137,8 @@ func applyDefaultConfig(conf *config.Configuration) {
 	}
 
 	// Importers
-	if conf.Importers.WebAPI.Endpoint == "" {
-		conf.Importers.WebAPI.Endpoint = "/sites"
-	}
-
-	if conf.Importers.AdminAPI.Endpoint == "" {
-		conf.Importers.AdminAPI.Endpoint = "/admin"
+	if conf.Importers.SiteRegistration.Endpoint == "" {
+		conf.Importers.SiteRegistration.Endpoint = "/sitereg"
 	}
 
 	// Exporters