Resolve Wikidata URIs via LDF endpoint and LC URIs using HEAD requests #921
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Issues
======
+ Solved 5
- Added 10
Complexity increasing per file
==============================
- model/ConceptPropertyValue.php 1
- model/resolver/Resolver.php 3
- model/resolver/LDFResource.php 2
- model/resolver/LOCResource.php 4
- model/resolver/LinkedDataResource.php 2
See the complete overview on Codacy |
osma
commented
Feb 5, 2020
| $res = new LOCResource($uri); | ||
| } elseif ($this->startsWith('http://www.wikidata.org/entity/', $uri)) { | ||
| $res = new LDFResource($uri, 'https://query.wikidata.org/bigdata/ldf'); | ||
| } else { |
There was a problem hiding this comment.
osma
commented
Feb 5, 2020
model/resolver/LDFResource.php
Outdated
| // change the timeout setting for external requests | ||
| $httpclient = EasyRdf\Http::getDefaultHttpClient(); | ||
| $httpclient->setConfig(array('timeout' => $timeout)); | ||
| EasyRdf\Http::setDefaultHttpClient($httpclient); |
There was a problem hiding this comment.
osma
commented
Feb 5, 2020
| // 1. Unregister the legacy RDF/JSON parser, we don't want to use it | ||
| EasyRdf\Format::unregister('json'); | ||
| // 2. Add "application/json" as a possible MIME type for the JSON-LD format | ||
| $jsonld = EasyRdf\Format::getFormat('jsonld'); |
There was a problem hiding this comment.
osma
commented
Feb 5, 2020
| // change the timeout setting for external requests | ||
| $httpclient = EasyRdf\Http::getDefaultHttpClient(); | ||
| $httpclient->setConfig(array('timeout' => $timeout)); | ||
| EasyRdf\Http::setDefaultHttpClient($httpclient); |
There was a problem hiding this comment.
osma
commented
Feb 5, 2020
|
|
||
| try { | ||
| // change the timeout setting for external requests | ||
| $httpclient = EasyRdf\Http::getDefaultHttpClient(); |
There was a problem hiding this comment.
osma
commented
Feb 5, 2020
model/resolver/LDFResource.php
Outdated
| public function resolve(int $timeout) : ?EasyRdf\Resource { | ||
| try { | ||
| // change the timeout setting for external requests | ||
| $httpclient = EasyRdf\Http::getDefaultHttpClient(); |
There was a problem hiding this comment.
osma
commented
Feb 5, 2020
| $httpclient->setConfig(array('timeout' => $timeout)); | ||
| EasyRdf\Http::setDefaultHttpClient($httpclient); | ||
|
|
||
| $graph = EasyRdf\Graph::newAndLoad(EasyRdf\Utils::removeFragmentFromUri($this->uri)); |
There was a problem hiding this comment.
osma
commented
Feb 5, 2020
| { | ||
| if ($this->isExternal()) { | ||
| return $this->vocab; | ||
| } else { |
There was a problem hiding this comment.
osma
commented
Feb 5, 2020
| public function resolve(int $timeout) : ?EasyRdf\Resource { | ||
| // prevent parsing errors for sources which return invalid JSON (see #447) | ||
| // 1. Unregister the legacy RDF/JSON parser, we don't want to use it | ||
| EasyRdf\Format::unregister('json'); |
There was a problem hiding this comment.
osma
commented
Feb 5, 2020
model/resolver/LOCResource.php
Outdated
| 'user_agent' => 'Skosmos', | ||
| 'timeout' => $timeout)); | ||
| $context = stream_context_create($opts); | ||
| $fd = fopen($this->uri, 'rb', false, $context); |
There was a problem hiding this comment.
Codecov Report
@@ Coverage Diff @@
## master #921 +/- ##
============================================
- Coverage 65.76% 59.69% -6.08%
- Complexity 1469 1488 +19
============================================
Files 27 32 +5
Lines 3637 4037 +400
============================================
+ Hits 2392 2410 +18
- Misses 1245 1627 +382
Continue to review full report at Codecov.
|
…andling property values that are outside the current vocabulary, but in another vocabulary known to Skosmos.
…relying on label heuristic
osma
force-pushed
the
issue912-lookup-wikidata-ldf
branch
from
264ce87 to
63932b8
Compare
|
Kudos, SonarCloud Quality Gate passed!
|
osma
added a commit
that referenced
this pull request
Feb 10, 2020
osma
added a commit
that referenced
this pull request
Feb 10, 2020
fix some test coverage annotations after PR #921
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR refactors and modularizes the lookup/resolution mechanism for remote URIs, that is, URIs that Skosmos knows nothing about.
There is a new Resolver class which knows which resolution mechanism to use for a particular URI. Wikidata URIs are looked up via the LDF endpoint (fixes #912); HEAD requests are used for LCSH and other LC vocabularies (fixes #915); and for everything else, a generic Linked Data mechanism (just retrieve the URI and ask for RDF) is used, as before.
Unfortunately, it turns out that the Wikidata LDF endpoint cannot currently handle language tags. I've reported this to the Wikidata list. If the issue isn't fixed soon, then that mechanism has to be disabled for now - perhaps the Wikidata SPARQL endpoint could be used instead, although it may be less reliable than the LDF endpoint.
I'm keeping this just as a draft PR for now, until there is more clarity on the LDF endpoint issue. I just want to see the automated QA reports.