No gemini #111

dannylamb · 2021-01-06T17:12:17Z

GitHub Issue: Part of Islandora-Devops/isle-dc#132

What does this Pull Request do?

Delete Gemini. Rewrites milliner to use code in crayfish-commons instead of calling out to Gemini.

How should this be tested?

Test with Islandora-Devops/isle-dc#132

Interested parties

@Islandora/8-x-committers

…rce field is no longer required.

elizoller · 2021-01-22T20:09:34Z

does this mean that no gemini = fedora 6?

dannylamb · 2021-01-28T20:51:28Z

@elizoller It's all jumbled up here. But they're not strictly requiring one or the other.

codecov · 2021-02-03T17:51:56Z

Codecov Report

Merging #111 (84fd764) into dev (291ebe0) will decrease coverage by 14.31%.
The diff coverage is 50.65%.

@@              Coverage Diff              @@
##                dev     #111       +/-   ##
=============================================
- Coverage     91.87%   77.55%   -14.32%     
+ Complexity      175      159       -16     
=============================================
  Files             9        6        -3     
  Lines           726      655       -71     
=============================================
- Hits            667      508      -159     
- Misses           59      147       +88

Impacted Files	Coverage Δ	Complexity Δ
Recast/src/Controller/RecastController.php	`55.19% <30.76%> (-19.15%)`	`48.00 <13.00> (+12.00)`	⬇️
Milliner/src/Service/MillinerService.php	`71.00% <52.11%> (-18.85%)`	`59.00 <13.00> (+3.00)`	⬇️
Homarus/src/Controller/HomarusController.php	`100.00% <100.00%> (ø)`	`13.00 <0.00> (ø)`
Milliner/src/Controller/MillinerController.php	`100.00% <100.00%> (+2.97%)`	`22.00 <6.00> (-2.00)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 44eaba6...84fd764. Read the comment docs.

dannylamb · 2021-02-03T17:55:00Z

Codecov is grumpy b/c deleting a whole microservice has upset the code coverage balance. But other than that this is green. Travis abides by all my updated tests.

dannylamb · 2021-03-05T20:14:08Z

@elizoller Took care of recast and its tests. Took longer than I expected/wanted, but it's there!

elizoller · 2021-03-05T20:24:18Z

cool, i can give it another test 👍

elizoller · 2021-03-05T21:31:15Z

i've pulled the most recents down on this branch in crayfish and did a composer install inside recast
now i am testing recast service:
curl -H"Authorization: Bearer islandora" -H"Accept: text/turtle" -H"Apix-Ldp-Resource: http://localhost:8080/fcrepo/rest/82/89/1a/3f/82891a3f-f23d-4561-95bd-9c4674c92695" http://localhost:8000/recast/
returns

@prefix fedora: <http://fedora.info/definitions/v4/repository#> .
@prefix ldp: <http://www.w3.org/ns/ldp#> .
@prefix pcdm: <http://pcdm.org/models#> .
@prefix schema: <http://schema.org/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix ns0: <http://id.loc.gov/vocabulary/relators/> .
@prefix dc: <http://purl.org/dc/terms/> .

<http://localhost:8080/fcrepo/rest/82/89/1a/3f/82891a3f-f23d-4561-95bd-9c4674c92695>
  a fedora:Container, <http://purl.org/coar/resource_type/c_c513>, fedora:Resource, ldp:BasicContainer, pcdm:Object, ldp:RDFSource, ldp:Container ;
  fedora:lastModifiedBy "bypassAdmin" ;
  schema:dateModified "2021-03-05T21:26:09+00:00"^^xsd:dateTime ;
  schema:author <http://localhost:8000/user/1> ;
  ns0:prv <http://localhost:8000/taxonomy/term/37?_format=jsonld>, <http://localhost:8000/taxonomy/term/36?_format=jsonld> ;
  schema:sameAs <http://localhost:8000/node/12> ;
  schema:dateCreated "2021-03-05T21:16:19+00:00"^^xsd:dateTime ;
  ns0:pht <http://localhost:8000/taxonomy/term/35?_format=jsonld> ;
  dc:date "2018-01-01"^^xsd:date, "2018-01-01" ;
  dc:extent "1 item" ;
  fedora:createdBy "bypassAdmin" ;
  fedora:lastModified "2021-03-05T21:26:10.037Z"^^xsd:dateTime ;
  fedora:created "2021-03-05T21:16:21.666Z"^^xsd:dateTime ;
  dc:subject <http://localhost:8000/taxonomy/term/34>, <http://localhost:8000/taxonomy/term/33> ;
  pcdm:memberOf <http://localhost:8000/node/17> ;
  dc:description "A sign pointing towards a nail salon."@en ;
  dc:title "Nails Nails Nails"@en .

and /add returns the same thing as far as i can see:
curl -H"Authorization: Bearer islandora" -H"Accept: text/turtle" -H"Apix-Ldp-Resource//localhost:8080/fcrepo/rest/82/89/1a/3f/82891a3f-f23d-4561-95bd-9c4674c92695" http://localhost:8000/recast/add

@prefix fedora: <http://fedora.info/definitions/v4/repository#> .
@prefix ldp: <http://www.w3.org/ns/ldp#> .
@prefix pcdm: <http://pcdm.org/models#> .
@prefix schema: <http://schema.org/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix ns0: <http://id.loc.gov/vocabulary/relators/> .
@prefix dc: <http://purl.org/dc/terms/> .

<http://localhost:8080/fcrepo/rest/82/89/1a/3f/82891a3f-f23d-4561-95bd-9c4674c92695>
  a fedora:Container, <http://purl.org/coar/resource_type/c_c513>, fedora:Resource, ldp:BasicContainer, pcdm:Object, ldp:RDFSource, ldp:Container ;
  fedora:lastModifiedBy "bypassAdmin" ;
  schema:dateModified "2021-03-05T21:26:09+00:00"^^xsd:dateTime ;
  schema:author <http://localhost:8000/user/1> ;
  ns0:prv <http://localhost:8000/taxonomy/term/37?_format=jsonld>, <http://localhost:8000/taxonomy/term/36?_format=jsonld> ;
  schema:sameAs <http://localhost:8000/node/12> ;
  schema:dateCreated "2021-03-05T21:16:19+00:00"^^xsd:dateTime ;
  ns0:pht <http://localhost:8000/taxonomy/term/35?_format=jsonld> ;
  dc:date "2018-01-01"^^xsd:date, "2018-01-01" ;
  dc:extent "1 item" ;
  fedora:createdBy "bypassAdmin" ;
  fedora:lastModified "2021-03-05T21:26:10.037Z"^^xsd:dateTime ;
  fedora:created "2021-03-05T21:16:21.666Z"^^xsd:dateTime ;
  dc:subject <http://localhost:8000/taxonomy/term/34>, <http://localhost:8000/taxonomy/term/33> ;
  pcdm:memberOf <http://localhost:8000/node/17> ;
  dc:description "A sign pointing towards a nail salon."@en ;
  dc:title "Nails Nails Nails"@en .

and same with replace

am i missing a step somewhere?

elizoller · 2021-03-09T20:35:24Z

^ @dannylamb

dannylamb · 2021-03-10T14:19:29Z

I had to do some shenanigans to ensure things work on both http and https. As a side effect, you have to configure the Drupal and Fedora base url's for Recast without a protocol. Otherwise it won't make the match and you'll see what you're experiencing.

elizoller · 2021-03-10T15:47:36Z

in /var/www/html/Crayfish/Recast/cfg/config.yaml i changed it to:

fedora_resource:
  base_url: 127.0.0.1:8080/fcrepo/rest
gemini_base_url: http://127.0.0.1:8000/gemini
drupal_base_url: 127.0.0.1:8000

then i restarted apache just to be safe
then i reran curl commands to Recast as above and I'm still not getting the memberOf relationship transformed into a fedora URI, just getting the drupal URI

dannylamb · 2021-03-10T19:46:39Z

Crud... cuz that should do it. When I get a moment to come up for air I'll take another look 😿

dannylamb · 2021-03-11T16:20:05Z

Ah yes, the old 'break the code to fix the tests'.... le sigh.

Oh well, at least I know what's going on now. I can fix this and at least have ansible deploying it, too.

elizoller · 2021-03-11T16:47:35Z

👀

elizoller · 2021-03-11T19:00:26Z

pulled in this commit and did ran curl -H"Authorization: Bearer islandora" -H"Accept: text/turtle" -H"Apix-Ldp-Resource: http://localhost:8080/fcrepo/rest/82/89/1a/3f/82891a3f-f23d-4561-95bd-9c4674c92695" http://localhost:8000/recast/add

received this:

@prefix fedora: <http://fedora.info/definitions/v4/repository#> .
@prefix ldp: <http://www.w3.org/ns/ldp#> .
@prefix pcdm: <http://pcdm.org/models#> .
@prefix schema: <http://schema.org/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix ns0: <http://id.loc.gov/vocabulary/relators/> .
@prefix dc: <http://purl.org/dc/terms/> .

<http://localhost:8080/fcrepo/rest/82/89/1a/3f/82891a3f-f23d-4561-95bd-9c4674c92695>
  a fedora:Container, <http://purl.org/coar/resource_type/c_c513>, fedora:Resource, ldp:BasicContainer, pcdm:Object, ldp:RDFSource, ldp:Container ;
  fedora:lastModifiedBy "bypassAdmin" ;
  schema:dateModified "2021-03-05T21:26:09+00:00"^^xsd:dateTime ;
  schema:author <http://localhost:8000/user/1> ;
  ns0:prv <http://localhost:8000/taxonomy/term/37?_format=jsonld>, <http://localhost:8000/taxonomy/term/36?_format=jsonld> ;
  schema:sameAs <http://localhost:8000/node/12> ;
  schema:dateCreated "2021-03-05T21:16:19+00:00"^^xsd:dateTime ;
  ns0:pht <http://localhost:8000/taxonomy/term/35?_format=jsonld> ;
  dc:date "2018-01-01"^^xsd:date, "2018-01-01" ;
  dc:extent "1 item" ;
  fedora:createdBy "bypassAdmin" ;
  fedora:lastModified "2021-03-05T21:26:10.037Z"^^xsd:dateTime ;
  fedora:created "2021-03-05T21:16:21.666Z"^^xsd:dateTime ;
  dc:subject <http://localhost:8000/taxonomy/term/34>, <http://localhost:8000/taxonomy/term/33> ;
  pcdm:memberOf <http://localhost:8000/node/17> ;
  dc:description "A sign pointing towards a nail salon."@en ;
  dc:title "Nails Nails Nails"@en .

The recast log has:

[2021-03-11 12:59:28] app.INFO: Matched route "{route}". {"route":"GET_operation","route_parameters":{"_controller":"recast.controller:recast","operation":"add","_route":"GET_operation"},"request_uri":"http://localhost:8000/recast/add","method":"GET"} []
[2021-03-11 12:59:28] app.DEBUG: Checking for guard authentication credentials. {"firewall_key":"default","authenticators":1} []
[2021-03-11 12:59:28] app.DEBUG: Checking support on guard authenticator. {"firewall_key":"default","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-03-11 12:59:28] app.DEBUG: Calling getCredentials() on guard authenticator. {"firewall_key":"default","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-03-11 12:59:28] crayfish.syn.jwt_authentication.DEBUG: Token: islandora [] []
[2021-03-11 12:59:28] app.DEBUG: Passing guard token information to the GuardAuthenticationProvider {"firewall_key":"default","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-03-11 12:59:28] crayfish.syn.jwt_authentication.INFO: Logged in with static token: admin [] []
[2021-03-11 12:59:28] app.INFO: Guard authentication successful! {"token":"[object] (Symfony\\Component\\Security\\Guard\\Token\\PostAuthenticationGuardToken: PostAuthenticationGuardToken(user=\"admin\", authenticated=true, roles=\"admin\"))","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-03-11 12:59:28] app.DEBUG: Guard authenticator set no success response: request continues. {"authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-03-11 12:59:28] app.DEBUG: Remember me skipped: it is not configured for the firewall. {"authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-03-11 12:59:28] app.DEBUG: > GET /recast/add [] []
[2021-03-11 12:59:28] app.INFO: Request to add resource. [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://localhost:8080/fcrepo/rest/82/89/1a/3f/82891a3f-f23d-4561-95bd-9c4674c92695 [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://fedora.info/definitions/v4/repository#Container [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://purl.org/coar/resource_type/c_c513 [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://fedora.info/definitions/v4/repository#Resource [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://www.w3.org/ns/ldp#BasicContainer [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://pcdm.org/models#Object [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://localhost:8000/user/1 [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://localhost:8000/taxonomy/term/37?_format=jsonld [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://localhost:8000/taxonomy/term/36?_format=jsonld [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://localhost:8000/node/12 [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://localhost:8000/taxonomy/term/35?_format=jsonld [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://localhost:8000/taxonomy/term/34 [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://localhost:8000/taxonomy/term/33 [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://localhost:8000/node/17 [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://www.w3.org/ns/ldp#RDFSource [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Looking for reverse URI for: http://www.w3.org/ns/ldp#Container [] []
[2021-03-11 12:59:28] app.DEBUG: is a Drupal URL [] []
[2021-03-11 12:59:28] app.DEBUG: Adding acl -> http://www.w3.org/ns/auth/acl# [] []
[2021-03-11 12:59:28] app.DEBUG: Adding fedora -> http://fedora.info/definitions/v4/repository# [] []
[2021-03-11 12:59:28] app.DEBUG: Adding memento -> http://mementoweb.org/ns# [] []
[2021-03-11 12:59:28] app.DEBUG: Adding pcdm -> http://pcdm.org/models# [] []
[2021-03-11 12:59:28] app.DEBUG: Adding pcdmuse -> http://pcdm.org/use# [] []
[2021-03-11 12:59:28] app.DEBUG: Adding webac -> http://fedora.info/definitions/v4/webac# [] []
[2021-03-11 12:59:28] app.DEBUG: < 200 [] []

elizoller · 2021-03-11T19:22:01Z

Is that what i should be seeing, because I still don't see it working as how i'd expect it - seeing the fedora URi instead of the drupal URI for the member of?

dannylamb · 2021-03-12T18:06:39Z

Let me do another spin up of the playbook PR and make sure everything is as it should be. Sry this is being such a pain to test.

elizoller

I was able to get recast working successfully with the playbook PR Islandora-Devops/islandora-playbook#197

elizoller

I was able to get recast working successfully with the playbook PR Islandora-Devops/islandora-playbook#197

elizoller · 2021-03-12T18:33:51Z

it'll need the branch of crayfish-commons changed back though https://github.com/Islandora/Crayfish/pull/111/files#diff-94360fcbfdaa0794f89079d0ed59a44bf07deb14fb01050e69055e595004cc8fR7

dannylamb · 2021-03-12T19:30:15Z

Composesr.json's have been reverted for both milliner and recast 👍

elizoller · 2021-03-12T19:30:33Z

do the composer.locks need to be updated?

dannylamb · 2021-03-12T19:37:07Z

probably. i'll do that real quick.

adam-vessey · 2021-04-14T17:27:19Z

Just noticed... composer.phar got committed here.

Suspecting it was accidental, but figuring it should be confirmed somewhere before throwing in a commit to remove it.

dannylamb · 2021-04-14T18:08:11Z

🤦‍♂️ Oh noes.... that one's on me.

I'll make a PR to get rid of it.

dannylamb added 11 commits December 18, 2020 14:57

Configurable domains

6e197ab

Bwahahahaha

be22b10

No gemini

c32a746

Nodes and taxonomy terms work. Media needs alapca reworking since sou…

fa66790

…rce field is no longer required.

Updating lockfile

26bf337

Working on media

5eb68fe

whoops

458be85

Working now

06ea21f

working out more kinks

4d13cae

Changes for Fedora 6

fcd4af5

Using X-State-Token with X-If-State-Token instead of Etags and If-Match

7112998

Coding standards

b06221a

dannylamb added 2 commits January 28, 2021 16:56

Updating lockfile

0a4e81b

Coding standards and tests passing again

e25e379

dannylamb marked this pull request as ready for review February 3, 2021 17:55

dannylamb added 3 commits March 1, 2021 16:05

Adding no-gemini branch to Recast

de33e10

Gutting gemini client from Recast

b2a83dc

Tests!

e2fa1df

dannylamb mentioned this pull request Mar 5, 2021

Removing gemini Islandora/islandora#822

Merged

Refixing after breaking it to fix tests lol

ae9a35c

elizoller approved these changes Mar 12, 2021

View reviewed changes

dannylamb added 2 commits March 12, 2021 15:27

Update composer.json

387f36e

Update composer.json

16b11b3

Updating lockfiles

84fd764

elizoller merged commit c324f54 into dev Mar 12, 2021

dannylamb mentioned this pull request Apr 14, 2021

Deleting composer.phar #112

Merged

whikloj deleted the no-gemini branch April 29, 2021 14:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

No gemini #111

No gemini #111

dannylamb commented Jan 6, 2021

elizoller commented Jan 22, 2021

dannylamb commented Jan 28, 2021

codecov bot commented Feb 3, 2021 •

edited

Loading

dannylamb commented Feb 3, 2021

dannylamb commented Mar 5, 2021

elizoller commented Mar 5, 2021

elizoller commented Mar 5, 2021

elizoller commented Mar 9, 2021

dannylamb commented Mar 10, 2021

elizoller commented Mar 10, 2021

dannylamb commented Mar 10, 2021

dannylamb commented Mar 11, 2021

elizoller commented Mar 11, 2021

elizoller commented Mar 11, 2021

elizoller commented Mar 11, 2021

dannylamb commented Mar 12, 2021

elizoller left a comment

elizoller left a comment

elizoller commented Mar 12, 2021

dannylamb commented Mar 12, 2021

elizoller commented Mar 12, 2021

dannylamb commented Mar 12, 2021

adam-vessey commented Apr 14, 2021

dannylamb commented Apr 14, 2021

No gemini #111

No gemini #111

Conversation

dannylamb commented Jan 6, 2021

What does this Pull Request do?

How should this be tested?

Interested parties

elizoller commented Jan 22, 2021

dannylamb commented Jan 28, 2021

codecov bot commented Feb 3, 2021 • edited Loading

Codecov Report

dannylamb commented Feb 3, 2021

dannylamb commented Mar 5, 2021

elizoller commented Mar 5, 2021

elizoller commented Mar 5, 2021

elizoller commented Mar 9, 2021

dannylamb commented Mar 10, 2021

elizoller commented Mar 10, 2021

dannylamb commented Mar 10, 2021

dannylamb commented Mar 11, 2021

elizoller commented Mar 11, 2021

elizoller commented Mar 11, 2021

elizoller commented Mar 11, 2021

dannylamb commented Mar 12, 2021

elizoller left a comment

Choose a reason for hiding this comment

elizoller left a comment

Choose a reason for hiding this comment

elizoller commented Mar 12, 2021

dannylamb commented Mar 12, 2021

elizoller commented Mar 12, 2021

dannylamb commented Mar 12, 2021

adam-vessey commented Apr 14, 2021

dannylamb commented Apr 14, 2021

codecov bot commented Feb 3, 2021 •

edited

Loading