Return resources in the API v2 simple ontology schema #833

benjamingeer · 2018-04-25T14:52:51Z

Refactor ReadResourcesSequenceV2 and its components, separating schema conversion from JSON-LD generation.
Enable ReadResourcesSequenceV2 and its components to return JSON-LD in the simple schema.
Use rdfs:label in responses instead of schema:name (requires a change in SALSAH 2).
Enable ResourcesRouteV2 to handle requests that ask for a response in either the simple or the complex schema, using an HTTP header (X-Knora-Accept-Schema) or a URL parameter (schema).
Fix the format of the string representation of dates in both schemas, so it conforms to StringFormatter.KnoraDateRegex.
Represent IRI values correctly in JSON-LD (Incorrect representation of IRI object values in JSON-LD #835), instead of as simple strings (requires a change in SALSAH 2).
Add tests.
Update docs.
Update release notes.

Resolves #725.
Fixes #835.
Fixes #797.

Requires dhlab-basel/Salsah#245 to be merged

…ma (ongoing). - Use rdfs:label in responses instead of schema:name.

…ma (ongoing).

# Conflicts: # webapi/src/test/resources/test-data/resourcesR2RV2/NarrenschiffFirstPage.jsonld # webapi/src/test/resources/test-data/resourcesR2RV2/NarrenschiffFirstPageExpanded.jsonld # webapi/src/test/resources/test-data/searchR2RV2/IncomingLinksForBook.jsonld # webapi/src/test/resources/test-data/searchR2RV2/RegionsForPage.jsonld # webapi/src/test/resources/test-data/searchR2RV2/ThingWithBooleanOptional.jsonld # webapi/src/test/scala/org/knora/webapi/responders/v2/ResourcesResponderV2SpecFullData.scala # webapi/src/test/scala/org/knora/webapi/responders/v2/SearchResponderV2SpecFullData.scala

- Refactor ResponseCheckerR2RV2 to use JsonLDDocument and to handle values in the simple schema.

- Add tests.

… it from the request.

benjamingeer · 2018-04-27T15:39:06Z

@lrosenth I've been thinking about how we can handle content negotiation when using ARK URLs for resources. I suggest it could work like this:

If the client is a browser, they'll probably send an HTTP Accept header like this when they open the ARK URL:

Accept: text/html

Then our ARK server can redirect them to the SALSAH GUI.

If the client wants JSON or JSON-LD, they'll probably send an Accept header like this:

Accept: application/json,application/ld+json

In that case, we could assume that they don't know much about Knora (otherwise they wouldn't be using the ARK URL), and that they want something like Linked Open Data, i.e. something simple and interoperable. So ARK server can redirect them directly to the Knora API to get the resource, and add this header:

X-Knora-Accept-Schema: simple

Then they'll get the resource as JSON-LD, using the simple schema.

How does that sound?

benjamingeer · 2018-04-27T15:40:49Z

...or instead of using X-Knora-Accept-Schema, the ARK server could just add ?schema=simple to the Knora API URL.

subotic · 2018-04-27T21:31:35Z

If I may. Sounds good to me. Are we going to support both X-Knora-Accept-Schema and ?schema=simple?

lrosenth · 2018-04-27T22:24:23Z

Sound very reasonable to me!! Thanks, Lukas Am 27.04.2018 um 17:39 schrieb Benjamin Geer <[email protected]<mailto:[email protected]>>: @lrosenth<https://github.com/lrosenth> I've been thinking about how we can handle content negotiation when using ARK URLs for resources. I suggest it could work like this: If the client is a browser, they'll probably send an HTTP Accept header like this when they open the ARK URL: Accept: text/html Then our ARK server can redirect them to the SALSAH GUI. If the client wants JSON or JSON-LD, they'll probably send an Accept header like this: Accept: application/json,application/ld+json In that case, we could assume that they don't know much about Knora (otherwise they wouldn't be using the ARK URL), and that they want something like Linked Open Data, i.e. something simple and interoperable. So ARK server can redirect them directly to the Knora API to get the resource, and add this header: X-Knora-Accept-Schema: simple Then they'll get the resource as JSON-LD, using the simple schema. How does that sound? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#833 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFN9zIb7ogRc1lqUOXPQsWoevDmG0dlrks5tszubgaJpZM4Tjlmm>.

benjamingeer · 2018-04-28T07:39:57Z

Sounds good to me. Are we going to support both X-Knora-Accept-Schema and ?schema=simple?

Both are supported in this PR.

- Fix representations of list values and links in JSON-LD in the simple schema. - Add more tests for resources returned as JSON-LD in the simple schema. - Fix incorrect tests. - Fix GenerateContributorsFile. - Update release notes.

# Conflicts: # docs/src/paradox/00-release-notes/v1.5.0.md

tobiasschweizer · 2018-05-08T11:31:05Z

Contributors.md

    - Stefan Münnich <https://github.com/musicEnfanthen>
+    - André Kilchenmann <https://github.com/kilchenmann>
+    - <https://github.com/janCstoffregen>


Shouldn't that be "Jan Stoffregen https://github.com/janCstoffregen"`

It can't be, because he hasn't put his name in his GitHub accont.

# Conflicts: # docs/src/sphinx/03-knora-api-server/api_v2/api-general.rst # docs/src/sphinx/03-knora-api-server/api_v2/knora-iris.rst # docs/src/sphinx/03-knora-api-server/api_v2/ontology-information.rst # docs/src/sphinx/03-knora-api-server/api_v2/reading-and-searching-resources.rst

# Conflicts: # webapi/src/main/scala/org/knora/webapi/messages/v2/responder/KnoraResponseV2.scala # webapi/src/main/scala/org/knora/webapi/util/ConstructResponseUtilV2.scala # webapi/src/test/scala/org/knora/webapi/e2e/v2/JSONLDHandlingV2R2RSpec.scala # webapi/src/test/scala/org/knora/webapi/e2e/v2/ResponseCheckerR2RV2.scala # webapi/src/test/scala/org/knora/webapi/responders/v2/SearchResponderV2SpecFullData.scala

tobiasschweizer · 2018-05-11T08:13:33Z

docs/src/paradox/03-webapi/api-v2/ontology-information.md

@@ -47,27 +47,45 @@ To get metadata about all ontologies:
 HTTP GET to http://host/v2/ontologies/metadata
 ```

-The response is in the default API v2 schema. Sample response:
+The response is in the complex API v2 schema. Sample response:



Do you want to add this to the TypeScript interfaces? -> docs/src/api-v2

You can have a look at resourcesResponse.ts as an example

Can I do this in a later pull request?

Absolutely, it i snot urgent.

tobiasschweizer · 2018-05-11T08:23:11Z

.../main/scala/org/knora/webapi/messages/v2/responder/ontologymessages/OntologyMessagesV2.scala

+                            )
+                        } else if (objArray.value.forall {
+                            case jsonObjElem: JsonLDObject if JsonLDUtil.isStringWithLang(jsonObjElem) =>
+                                // All the elements of the array are strings with language codes.


Are these language codes going to end up in the response? Or is there always only one language returned to the client (preferred language or fallback language)?

This code is processing JSON-LD received from the client in a request. It is not generating JSON-LD to be sent to the client.

When an ontology is created or modified?

When an ontology entity is created or modified.

Ok, got it.

tobiasschweizer · 2018-05-11T11:05:05Z

webapi/src/test/scala/org/knora/webapi/e2e/v2/OntologyV2R2RSpec.scala

@@ -270,46 +266,16 @@ class OntologyV2R2RSpec extends R2RSpec {
            }
        }

-        "serve a project-specific ontology whose IRI contains a project ID, as JSON-LD, using the simple schema" in {


Why did you remove these test?

It became redundant, because now all the ontology IRIs contain project IDs. So this is already covered by the tests that serve the incunabula ontology.

Ok, I understand.

tobiasschweizer · 2018-05-11T11:10:17Z

webapi/src/test/scala/org/knora/webapi/e2e/v2/ResourcesRouteV2R2Spec.scala

@@ -93,6 +94,32 @@ class ResourcesRouteV2R2Spec extends R2RSpec {
            }
        }

+        "perform a resource request for the book 'Reise ins Heilige Land' using the simple schema (specified by an HTTP header)" in {
+
+            Get(s"/v2/resources/${URLEncoder.encode("http://rdfh.ch/2a6221216701", "UTF-8")}").addHeader(new SchemaHeader(RouteUtilV2.SIMPLE_SCHEMA_NAME)) ~> resourcesPath ~> check {


addHeader(new SchemaHeader(RouteUtilV2.SIMPLE_SCHEMA_NAME)

Wouldn't it be better for a test to send the complete header as a string: "x-knora-accept-schema: simple"

In order to make sure that the parsing works correctly.

There is no way to do that with Akka-HTTP. A custom header has to be specified as a class. There's a link to the relevant docs in SchemaHeader.scala.

I think this is because the tests are not actual HTTP requests as the browser would send them.

This isn't anything specific to testing, it seems to be inherent in the design of the Akka HTTP client API:

https://doc.akka.io/docs/akka-http/current/common/http-model.html#custom-headers

I didn't find any other way to add a custom header to an HTTP request in Akka.

tobiasschweizer · 2018-05-11T11:13:23Z

webapi/src/test/scala/org/knora/webapi/e2e/v2/ResponseCheckerR2RV2.scala

-
-        assert(expectedResponseAsScala == receivedResponseAsScala, "Mapping creation request response did not match expected response")
-
+        assert(expectedJsonLDDocument == receivedJsonLDDocument, "Mapping creation request response did not match expected response")


Should we add a test to make sure this method works correctly?

We could add a test the expects this method to fail. I am thinking of the received == received or expected == expcted bug ...

I know there is no logical way to rule out errors. So I won't ask for a test that checks the tests for the test ...

OK, I'll add a ResponseCheckerV2Spec.

Great, thanks!

feature (webapi): Return resources in the API v2 simple ontology sche…

d3f9d55

…ma (ongoing). - Use rdfs:label in responses instead of schema:name.

benjamingeer added enhancement improve existing code or new feature API/V2 breaking (salsah2) labels Apr 25, 2018

benjamingeer added this to the v1.5.0 milestone Apr 25, 2018

Benjamin Geer added 7 commits April 26, 2018 11:36

feature (webapi): Return resources in the API v2 simple ontology sche…

b786317

…ma (ongoing).

docs (webapi): Update docs and release notes.

991c14d

feature (webapi): Return resources in the API v2 simple ontology sche…

132c2ab

…ma (ongoing).

test (webapi): Add test that gets a resource in the simple schema.

ad1a87f

- Refactor ResponseCheckerR2RV2 to use JsonLDDocument and to handle values in the simple schema.

feature (webapi): Enable the simple schema in search responses.

7edf7f3

- Add tests.

refactor (webapi): Require routes to specify a response schema or get…

4f105e8

… it from the request.

benjamingeer requested a review from tobiasschweizer April 27, 2018 12:55

benjamingeer assigned benjamingeer and tobiasschweizer Apr 27, 2018

benjamingeer added the ready-for-review label Apr 27, 2018

benjamingeer added in-progress and removed ready-for-review labels Apr 30, 2018

Benjamin Geer added 3 commits April 30, 2018 14:32

fix (webapi): Represent IRI values correctly in JSON-LD (#835).

ef79950

Merge branch 'develop' into wip/return-resources-in-v2-simple

7134c45

benjamingeer mentioned this pull request Apr 30, 2018

GenerateContributorsFile broken #797

Closed

Merge branch 'develop' into wip/return-resources-in-v2-simple

758e5d9

# Conflicts: # docs/src/paradox/00-release-notes/v1.5.0.md

benjamingeer added ready-for-review and removed in-progress labels Apr 30, 2018

tobiasschweizer reviewed May 8, 2018

View reviewed changes

Merge branch 'develop' into wip/return-resources-in-v2-simple

f61eb19

tobiasschweizer mentioned this pull request May 8, 2018

Entity Iris are now objects in JSON-LD dhlab-basel/Salsah#245

Merged

Benjamin Geer and others added 7 commits May 8, 2018 17:04

Merge branch 'develop' into wip/return-resources-in-v2-simple

6b19258

docs (typescript): adapt typescript interfaces

bead959

docs (typescript): adapt sample data

90d4b51

docs (simple schema): document simple schema and add test data

53f83c9

fix (list Iris in anything data): fix wrong Iris

6ccdbd4

tobiasschweizer reviewed May 11, 2018

View reviewed changes

test (test): Add test of test.

483bfb0

tobiasschweizer approved these changes May 11, 2018

View reviewed changes

benjamingeer merged commit f618eae into develop May 11, 2018

benjamingeer deleted the wip/return-resources-in-v2-simple branch May 11, 2018 12:46

benjamingeer mentioned this pull request Jul 5, 2019

Ben's PR history #571

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Return resources in the API v2 simple ontology schema #833

Return resources in the API v2 simple ontology schema #833

benjamingeer commented Apr 25, 2018 •

edited by tobiasschweizer

Loading

benjamingeer commented Apr 27, 2018

benjamingeer commented Apr 27, 2018

subotic commented Apr 27, 2018

lrosenth commented Apr 27, 2018 via email

benjamingeer commented Apr 28, 2018

tobiasschweizer May 8, 2018

benjamingeer May 8, 2018

tobiasschweizer May 11, 2018

benjamingeer May 11, 2018

tobiasschweizer May 11, 2018

tobiasschweizer May 11, 2018

benjamingeer May 11, 2018

tobiasschweizer May 11, 2018

benjamingeer May 11, 2018

tobiasschweizer May 11, 2018

tobiasschweizer May 11, 2018

benjamingeer May 11, 2018

tobiasschweizer May 11, 2018

tobiasschweizer May 11, 2018

benjamingeer May 11, 2018

tobiasschweizer May 11, 2018

benjamingeer May 11, 2018

tobiasschweizer May 11, 2018

tobiasschweizer May 11, 2018

benjamingeer May 11, 2018

tobiasschweizer May 11, 2018


		assert(expectedResponseAsScala == receivedResponseAsScala, "Mapping creation request response did not match expected response")

		assert(expectedJsonLDDocument == receivedJsonLDDocument, "Mapping creation request response did not match expected response")

Return resources in the API v2 simple ontology schema #833

Return resources in the API v2 simple ontology schema #833

Conversation

benjamingeer commented Apr 25, 2018 • edited by tobiasschweizer Loading

benjamingeer commented Apr 27, 2018

benjamingeer commented Apr 27, 2018

subotic commented Apr 27, 2018

lrosenth commented Apr 27, 2018 via email

benjamingeer commented Apr 28, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjamingeer commented Apr 25, 2018 •

edited by tobiasschweizer

Loading