Redesign the ontology responder for better performance #252
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
September 5, 2016 20:17
…ter performance (#185) (WIP).
| @@ -2283,7 +2284,9 @@ | |||
|
|
|||
| :resourceIcon "region.gif"^^xsd:string ; | |||
|
|
|||
| :hasPermissions "V knora-base:UnknownUser,knora-base:KnownUser|M knora-base:ProjectMember" ; | |||
There was a problem hiding this comment.
This is the state I have on my test server for BEOL. What happens if someone creates a Region now? Will it have any permissions?
There was a problem hiding this comment.
No, it will have no permissions.
… changes - fix broken indices in browers tests - prevent hasStandoffLinkTo from being submitted on resource creation
|
I think this should be excluded since it is a generic property that has to be used via subproperties only. |
added 2 commits
September 9, 2016 14:51
…by the ontology responder.
- Change SALSAH tests to refresh the ontology cache before running.
benjamingeer
pushed a commit
that referenced
this pull request
Oct 25, 2016
* feature (salsah): Start implementing rdfs:label as a pseudo-value. * feature (salsah): Add a special property for viewing and editing rdfs:label in different resource viewers. * feature (change a resource's label): add a route to change a resource's label - TODO: documentation and tests * feature (change a resource's label): add suppor in the GUI for changing a label - TODO: reset does not work properly yet. Editing is still active. * test (change a resource's label): add tests and documentation * fix (change a resource's label): clarify comments, improve SPARQL * fix (change a resource's label): indicate named graph in update query - remove resource class check from update query's where clause (because it is in the knora-base named graph) * enhancement (SPARQL update): check that resource is a subclass of kb:Resource - use GRAPH statement in DELETE and INSERT clauses instead of global WITH * fix (webapi): Fix default triplestore setting. * test: Fix broken test. * fix (salsah): Add support for creating a new resource with a label (WIP) * fix (webapi): Implement property predicate inheritance, which was missing from #252 and caused #263. * docs: Clarify rdfs:label in knora-base.tex. * refactor (webapi): Simplify property predicate inheritance. * style (webapi): Fix typo. * fix (salsah): Don't submit invalid JSON for an optional property that that user removed from the resource creation form (#179). * fix (salsah): Don't allow a region to be created in jquery.resadd.js. - Sort property definitions by guiorder in the API response describing a resource type. - Don't display an add button for knora-base:hasStandoffLinkto. - Try to display the label on a region (not working yet). * fix (salsah): Support knora-base:UriValue. * feature (salsah): Add support for boolean values. * test: Fix test data now that the ontology responder sorts properties by guiorder. * feature (salsah): Fix label and comment editing and display (#261 and #284) (in progress) - Display labels on regions. - If there is no comment on a value, return a JSON null in the API. - Add a route 'valuecomments' with an HTTP delete method for deleting a comment. * feature (webapi): Merge changes from #285 to support optional value comments. * docs: Update knora-base.rst. * fix (salah): Fix editing a comment after deleting it. - Fix value responder test of comment editing. - Add missing SALSAH icons. * fix (salsah): Don't allow the user to write a comment on a label. * test: Fix SALSAH resource creation test. * docs: Fix typescript interfaces to allow nulls. * fix (webapi): Return a value's comment when querying the value. - Add tests for creating and querying value comments.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request implements a new loading and caching design in the ontology responder.
The old design was based on the assumption that resource and property definitions could be loaded and cached incrementally, and that since they were cached, they could be loaded via relatively complex (hence relatively slow) SPARQL queries. This proved to be too slow in the case where all definitions must be loaded and returned for a single API request (#185).
The new design loads and caches all ontology information when the application starts. It uses simple, fast SPARQL queries, and does the rest of the processing in Scala, e.g. to calculate cardinality inheritance in resource classes.
To make this work, the ontology responder needs to receive a
LoadOntologiesRequestwhen the application starts.KnoraServicesends this message on startup, and I also changed each test spec to send it after the test data is loaded.Previously, the data in an
EntityInfoV1was pre-filtered according to the user's preferred language and the system's default language. Now, anEntityInfoV1contains all available translations, and the methodgetPredicateObjectcan be used to filter by language.I fixed some bugs in
knora-base.ttl:hasPermissionswas attached to some resource classes, but it is only for resource instances.knora-dc.ttlas well as full definitions inknora-base.ttl, but Knora assumes that each class is defined in only one ontology.I also fixed some incorrect test data.
Closes #185.