Use salsah-gui:guiOrder in cardinalities instead of in property definitions #737

benjamingeer · 2018-02-13T20:55:23Z

Overview:

Attach salsah-gui:guiOrder to cardinalities in class definitions, instead of attaching it to property definitions. This allows the same property to be first in one class and last in another class.
Validate the objects of salsah-gui:guiAttribute.

Tasks:

Manually update existing test ontologies (except for drawings-gods_ontology.ttl) so they use salsah-gui:guiOrder on cardinalities rather than on properties.
Provide a command-line program that updates salsah-gui:guiOrder in an existing ontology, and use it to update drawings-gods_ontology.ttl, which is too large to update manually.
Make existing functionality work in API v1 and v2.
Show salsah-gui:guiOrder in cardinalities in class definitions returned by the API v2 ontology routes.
Allow the client to specify salsah-gui:guiOrder when updating cardinalities via API v2.
Redesign salsah-gui:guiAttribute and salsah-gui:guiAttributeDefinition as a DSL parsed by StringFormatter, and use the attribute definitions to validate the attributes when loading ontologies from the triplestore.
Fix incorrect uses of salsah-gui:guiAttribute in test ontologies.
Document the salsah-gui ontology (reorganising and updating the Knora ontology docs a bit) and the rest of the above.
Update release notes.

TODO in a subsequent pull request:

Show salsah-gui:guiElement and salsah-gui:guiAttribute when serving ontologies in API v2.
Allow salsah-gui:guiElement and salsah-gui:guiAttribute to be updated via API v2.

Resolves #601.
Resolves #744.

…(ongoing).

musicEnfanthen · 2018-02-14T10:22:40Z

This is a move highly appreciated, thanks for that!

Would it make sense to move also the salsah-gui:guiElement to the cardinality restrictions in class definitions, so the property definitions are "free" of gui specific values? (Having a certain gui representation should’nt be considered a semantic characteristic of a property.) Thus, one could also have a property to be richtext in one class or just simpletext in another class, if desired.

mrivoal · 2018-02-14T11:36:49Z

@musicEnfanthen I can see your point, but I think it is a bit less relevant than having the salsah-gui:gui-order defined in class definition. And it would make the process of defining a resource very time-consuming if for a property used in multiple resource you have to specify for each resource the salsah-gui:guiElement of a given property.

I would rather define once and for all that a title in a given ontology is a salsah-gui:guiElement salsah-gui:SimpleText.

benjamingeer · 2018-02-14T11:54:20Z

If we support salsah-gui:guiElement in cardinalities, we'd have to support salsah-gui:guiAttribute there, too. I could imagine supporting both options. Perhaps SALSAH 2 could use the values on the cardinality if provided, otherwise it would look for ones on the property itself.

But putting more stuff in cardinalities is awkward with the current design in the ontology responder. If I do that, I'd like to do it later as part of #738.

…ities.

…tology. - Update drawings-gods_ontology.ttl.

…ons. - Still lots more inconsistent test data to fix.

…iAttributeDefinition. - Add more documentation. - Fix invalid GUI attributes in test ontologies and in knora-base. - Fix tests.

benjamingeer · 2018-02-23T10:33:05Z

@subotic you can review whenever you like :-)

subotic

While reading the salsah-gui.rst it seems to me that this ontology is very Salsah-V1 specific.

Wouldn't something like this :guiAttributeDefinition "hlist(required):iri" be better broken down into individual properties?

subotic · 2018-02-23T13:01:09Z

docs/rst/02-knora-ontologies/knora-base.rst

@@ -443,7 +336,7 @@ Properties of Value
    A resource may have several properties of the same type with
    different values (which will be of the same class), and it may be
    necessary to indicate an order in which these values occur. For
-    example, a book may have several authors which should appear in a
+    example, a book may have several authors which should appear in af


typo at af?

subotic · 2018-02-23T13:04:08Z

docs/rst/02-knora-ontologies/knora-base.rst

@@ -970,7 +863,7 @@ Knora's built-in support for “standoff” markup, which is stored
 separately from the text. This has some advantages over embedded markup
 such as XML. [#]_ While XML requires markup to have a hierarchical
 structure, and does not allow overlapping tags, standoff nodes do not
-have these limitations (Schmidt2016_). A standoff tag can be attached to
+have these limitations. [Schmidt2016]_ A standoff tag can be attached to


Shouldn't the link be before the period?

It's a footnote, so I just tried to format it like the other footnotes in that document. Apparently it was syntactically invalid before, but older versions of Sphinx never complained. When I upgraded Sphinx, it told me there was an error on that line. That's the only reason I changed it.

benjamingeer · 2018-02-23T14:04:22Z

While reading the salsah-gui.rst it seems to me that this ontology is very Salsah-V1 specific.

How so? Won't SALSAH 2 also use HTML elements for input?

Wouldn't something like this :guiAttributeDefinition "hlist(required):iri" be better broken down into individual properties?

What do you mean by individual properties? Can you give some examples?

All I tried to do here was document and enforce the conventions that are already used in project-specific ontologies. Keep in mind that if we change those conventions, we also have to change all project-specific ontologies.

subotic · 2018-02-23T14:42:22Z

I was just thinking out loud. Nothing that we should change now.

What do you mean by individual properties? Can you give some examples?

Something like this:

:hasListItem rdf:type owl:ObjectProperty ;
   salsah-gui:guiElement salsah-gui:List ;
   salsah-gui:guiHlistAttribute <http://rdfh.ch/lists/xyz> .

This would be somewhat safer, but would also require specifying a lot more properties in a lot more detail. Also, as you said, we would need to change a lot of existing data. This is probably to much.

benjamingeer · 2018-02-23T14:47:09Z

This would be somewhat safer

I agree, and I thought about this too, but I think it just isn't worth going to the trouble to change everyone's ontologies now. At least with this PR, the attributes are checked for validity.

subotic · 2018-02-23T15:41:18Z

webapi/src/test/resources/test-data/ontologyR2RV2/bildSimple.json

@@ -12,79 +12,93 @@
        "rdfs:subClassOf" : [ "http://api.knora.org/ontology/knora-api/simple/v2#StillImageRepresentation", {
          "@type" : "owl:Restriction",
          "owl:cardinality" : 1,
-          "owl:onProperty" : "http://0.0.0.0:3333/ontology/00FF/images/simple/v2#bearbeiter"
+          "owl:onProperty" : "http://api.knora.org/ontology/knora-api/simple/v2#creationDate"


Just a general question, where is this URL defined? I didn't find it in application.conf. I don't think that this a good idea, as this would mean, that we would need to have something running or serving at this address? Why not have it like this: `http://0.0.0.0:3333/ontology/knora-api/simple/v2#creationDate"?

This is documented here:

http://www.knora.org/documentation/manual/rst/03-knora-api-server/api_v2/knora-iris.html#external-ontology-iris

The idea is that we will have a Knora instance serving the built-in ontologies at that address.

If it was http://0.0.0.0:3333/ontology/knora-api/simple/v2#creationDate, then the knora-api ontology running on project-specific server A would not appear to be the same as the knora-api ontology running on project-specific server B. From a client's point of view, they would not be the same ontology so would not be interoperable.

Would it be generally possible to "dump" the knora-api ontologies to files? I'm thinking of an easy way to serve all our ontologies. If all the ontologies would be available as files, then it wouldn't be difficult to serve them.

The knora-api ontologies are already served by the Knora API server. You can request them from the API, download them, save them in files. That's what I did to make the test data for OntologyV2R2RSpec. It uses these files:

The knora-api v2 ontology in the simple schema: https://github.com/dhlab-basel/Knora/blob/develop/webapi/src/test/resources/test-data/ontologyR2RV2/knoraApiOntologySimple.json

The knora-api v2 ontology in the default schema: https://github.com/dhlab-basel/Knora/blob/develop/webapi/src/test/resources/test-data/ontologyR2RV2/knoraApiWithValueObjects.json

Currently the API server only serves ontologies in JSON-LD. What I've been planning to do is get the API server to take an HTTP Accept header and then be able to convert the JSON-LD to Turtle, XML, or HTML.

With the current design, we can't use www.knora.org for external (API v2) ontologies, because it's reserved for internal ontologies (the ones used in the triplestore):

http://www.knora.org/documentation/manual/rst/03-knora-api-server/api_v2/knora-iris.html#internal-ontology-iris

SmartIriImpl uses the hostname to determine whether the IRI is internal or external.

We could redesign all this, but I'd really rather not. :)

Thanks for bearing with me :-)

No problem. :)

Also, I don't think we should rely on GitHub to serve Knora ontologies. That would mean that if GitHub is down, the DaSCH is down. Also, my understanding is that as critical research infrastructure for Switzerland, the DaSCH is supposed to be hosted in Switzerland.

The less dependency there is on a running Knora stack

The DaSCH can't exist without a running Knora stack.

We could redesign all this, but I'd really rather not. :)

ok, we should then leave it as it is.

The less dependency there is on a running Knora stack

The DaSCH can't exist without a running Knora stack.

What I meant by this is, if our Knora stack is down for a few hours (which is acceptable by DaSCH's / Lukas's standard) then this shouldn't have a negative impact on all other running Knora instances around Switzerland / World because they all depend on our served ontologies.

That's why I would like to see a separation of the serving of ontologies from the Knora stack and implement the serving of instances in a lightweight and highly available (redundant) way. Nginx serving flat files is a much simpler and lightweight (read costs less) setup, then the whole Knora stack.

We will host the DaSCH Knora stack with Switch, but not completely in a high availability way. The data will have a redundancy of 6 with data in Zürich and Lausanne (losing data is not acceptable), but the machine running the stack will not be redundant (few hours of downtime acceptable).

I would love to have everything highly redundant so that we don't have a minute of downtime, but the budget constraints are what they are. Well, maybe someday this will change.

OK! I like flat files.

subotic

Did you maybe run the Salsah Gui tests?

benjamingeer · 2018-02-23T15:58:44Z

I would love to run the SALSAH GUI tests but for that I need a working Sipi. I seem to remember there was a nice Docker image for it, but I can't seem to find it now, is there some documentation I could use?

subotic · 2018-02-23T16:08:15Z

I personally always go to .travis.yml for the documentation:

# Salsah browser tests
    #- stage: test
    #  script:
    #    # prepare needed graphdb-se files
    #    - mkdir -p $TRAVIS_BUILD_DIR/data/conf
    #    - cp $TRAVIS_BUILD_DIR/private/graphdb.license $TRAVIS_BUILD_DIR/data/conf/graphdb.license
    #    - mkdir -p $TRAVIS_BUILD_DIR/graphdb
    #    - cp $TRAVIS_BUILD_DIR/webapi/scripts/KnoraRules.pie $TRAVIS_BUILD_DIR/graphdb
    #    # start and initialize graphdb-se
    #    - docker run -d -p 127.0.0.1:7200:7200 -v $TRAVIS_BUILD_DIR/data:/opt/graphdb/home -v $TRAVIS_BUILD_DIR/graphdb:/graphdb ontotext/graphdb:8.2.0-se
    #    - sleep 15
    #    - cd $TRAVIS_BUILD_DIR/webapi/scripts && ./graphdb-se-docker-init-knora-test.sh
    #    # start the sipi container (with test config)
    #    - docker run -d --add-host webapihost:$DOCKERHOST -v /tmp:/tmp -v $HOME:$HOME -p 1024:1024 $SIPI_REPO:develop /sipi/local/bin/sipi --config=/sipi/config/sipi.knora-test-docker-config.lua
    #    # start the webapi and salsah server
    #    - cd $TRAVIS_BUILD_DIR/webapi/ && sbt "run allowReloadOverHTTP" &
    #    - cd $TRAVIS_BUILD_DIR/salsah/ && sbt run &
    #    # wait for the webapi server to start before we proceed with running the salsah tests
    #    - until netstat -an 2>/dev/null | grep '3333.*LISTEN'; do true; done
    #    # unzip chromedriver
    #    - cd $TRAVIS_BUILD_DIR/salsah/lib/chromedriver/ && unzip chromedriver_linux64.zip
    #    # start built-in salsah browser tests
    #    - cd $TRAVIS_BUILD_DIR/salsah/ && sbt headless:test

I should probably add this to the Knora documentation.

subotic · 2018-02-23T16:11:19Z

for sipi, this is the relevant line:

$ docker run -d --add-host webapihost:$DOCKERHOST -v /tmp:/tmp -v $HOME:$HOME -p 1024:1024 $SIPI_REPO:develop /sipi/local/bin/sipi --config=/sipi/config/sipi.knora-test-docker-config.lua

$DOCKERHOST would be the IP address of your machine.
$SIPI_REPO would be dhlabbasel/sipi

benjamingeer · 2018-02-23T16:12:14Z

Cool, thanks!

I should probably add this to the Knora documentation.

That would be great. :)

subotic · 2018-02-23T23:17:20Z

Did you manage to get sipi working?

subotic · 2018-02-23T23:18:46Z

When I finish PR #742 it should be easy to run the whole stack.

benjamingeer · 2018-02-24T03:02:52Z

I ran out of time to try Sipi this week. Will try on Monday.

subotic

If the Salsah GUI tests pass, then please merge.

benjamingeer · 2018-02-26T16:23:49Z

I fixed the SALSAH GUI tests by adding a half-second delay to the login method. :)

benjamingeer · 2018-02-26T16:55:07Z

@subotic thanks for the review!

Benjamin Geer added 2 commits February 13, 2018 16:18

feature (webapi): Move salsah-gui:guiOrder into cardinalities (#601) …

279f86b

…(ongoing).

feature (webapi): Move salsah-gui:guiOrder into cardinalities (#601) …

99e5214

…(ongoing).

benjamingeer added enhancement improve existing code or new feature triplestore user requirement API/V1 API/V2 labels Feb 13, 2018

benjamingeer self-assigned this Feb 13, 2018

Merge branch 'develop' into wip/gui-order-in-cardinalities

d09e86a

benjamingeer added this to the v1.2.0 milestone Feb 13, 2018

Benjamin Geer added 2 commits February 14, 2018 17:27

test (webapi): Add tests for updating salsah-gui:guiOrder in cardinal…

a5e0b65

…ities.

feature (webapi): Add utility for moving salsah-gui:guiOrder in an on…

d9bd4f8

…tology. - Update drawings-gods_ontology.ttl.

benjamingeer mentioned this pull request Feb 16, 2018

salsah-gui:guiAttributeDefinition is undocumented and inconsistent #744

Closed

Benjamin Geer added 3 commits February 16, 2018 11:02

docs (knora-ontologies): Start documenting salsah-gui (#744).

a740f89

docs (knora-ontologies): Update release notes.

a76ce5b

feature (webapi): Validate salsah-gui attributes in property definiti…

e2613bb

…ons. - Still lots more inconsistent test data to fix.

benjamingeer added the ready-for-review label Feb 16, 2018

feature (webapi): Validate salah-gui:guiAttribute using salsah-gui:gu…

a321b4c

…iAttributeDefinition. - Add more documentation. - Fix invalid GUI attributes in test ontologies and in knora-base. - Fix tests.

benjamingeer requested a review from subotic February 17, 2018 00:05

Merge branch 'develop' into wip/gui-order-in-cardinalities

6324072

benjamingeer mentioned this pull request Feb 20, 2018

Include guiElement and guiAttribute in API v2 #746

Closed

Merge branch 'develop' into wip/gui-order-in-cardinalities

ac358a0

Merge branch 'develop' into wip/gui-order-in-cardinalities

676dbe7

benjamingeer removed the in-progress label Feb 23, 2018

fix (webapi): Fix comment.

57bf2de

subotic reviewed Feb 23, 2018

View reviewed changes

docs (knora-ontologies): Fix typo.

e7d8af3

subotic reviewed Feb 23, 2018

View reviewed changes

fix (webapi): Fix comment.

2937ab5

subotic approved these changes Feb 24, 2018

View reviewed changes

Benjamin Geer added 2 commits February 26, 2018 16:30

Merge branch 'develop' into wip/gui-order-in-cardinalities

a885d37

test (salsah): Fix tests.

c1045c8

benjamingeer merged commit c6e558e into develop Feb 26, 2018

benjamingeer deleted the wip/gui-order-in-cardinalities branch February 26, 2018 16:54

mrivoal mentioned this pull request Mar 7, 2018

Include guiElement and guiAttribute in API v2 #755

Merged

9 tasks

benjamingeer mentioned this pull request Jul 5, 2019

Ben's PR history #571

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use salsah-gui:guiOrder in cardinalities instead of in property definitions #737

Use salsah-gui:guiOrder in cardinalities instead of in property definitions #737

benjamingeer commented Feb 13, 2018 •

edited

Loading

musicEnfanthen commented Feb 14, 2018

mrivoal commented Feb 14, 2018

benjamingeer commented Feb 14, 2018

benjamingeer commented Feb 23, 2018

subotic left a comment

subotic Feb 23, 2018

benjamingeer Feb 23, 2018

subotic Feb 23, 2018

benjamingeer Feb 23, 2018

benjamingeer commented Feb 23, 2018

subotic commented Feb 23, 2018

benjamingeer commented Feb 23, 2018

subotic Feb 23, 2018

benjamingeer Feb 23, 2018

subotic Feb 23, 2018

benjamingeer Feb 23, 2018

benjamingeer Feb 23, 2018

benjamingeer Feb 24, 2018

benjamingeer Feb 24, 2018

benjamingeer Feb 24, 2018

subotic Feb 24, 2018

benjamingeer Feb 24, 2018

subotic left a comment

benjamingeer commented Feb 23, 2018

subotic commented Feb 23, 2018

subotic commented Feb 23, 2018 •

edited

Loading

benjamingeer commented Feb 23, 2018

subotic commented Feb 23, 2018

subotic commented Feb 23, 2018

benjamingeer commented Feb 24, 2018

subotic left a comment

benjamingeer commented Feb 26, 2018

benjamingeer commented Feb 26, 2018

Use salsah-gui:guiOrder in cardinalities instead of in property definitions #737

Use salsah-gui:guiOrder in cardinalities instead of in property definitions #737

Conversation

benjamingeer commented Feb 13, 2018 • edited Loading

musicEnfanthen commented Feb 14, 2018

mrivoal commented Feb 14, 2018

benjamingeer commented Feb 14, 2018

benjamingeer commented Feb 23, 2018

subotic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjamingeer commented Feb 23, 2018

subotic commented Feb 23, 2018

benjamingeer commented Feb 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

subotic left a comment

Choose a reason for hiding this comment

benjamingeer commented Feb 23, 2018

subotic commented Feb 23, 2018

subotic commented Feb 23, 2018 • edited Loading

benjamingeer commented Feb 23, 2018

subotic commented Feb 23, 2018

subotic commented Feb 23, 2018

benjamingeer commented Feb 24, 2018

subotic left a comment

Choose a reason for hiding this comment

benjamingeer commented Feb 26, 2018

benjamingeer commented Feb 26, 2018

benjamingeer commented Feb 13, 2018 •

edited

Loading

subotic commented Feb 23, 2018 •

edited

Loading