Removing gemini #822

dannylamb · 2021-01-08T18:46:37Z

JIRA Ticket: Goes with Islandora-Devops/isle-dc#132

What does this Pull Request do?

Uses the new EntityMapper service from Islandora/Crayfish-Commons#45 to recreate what GeminiLookup did. Deletes a lot of stuff 🔥

How should this be tested?

Pull in this PR to your Islandora module
Pull in No gemini Crayfish-Commons#45 to crayfish-commons in your sites vendor/islandora/crayfish-commons directory.
Clear the cache
Make a repository item and give it an Original File
Confirm the Fedora URL is correct for
- The node
- The Original File media (which lives in Fedora)
- A derivative (which does not live in Fedora)
- A taxonomy term (I looked at the Model of the node)

Interested parties

@Islandora/8-x-committers

elizoller · 2021-01-11T20:57:06Z

@dannylamb is this ready for testing?

… no-gemini

…to no-gemini

dannylamb · 2021-02-04T14:22:05Z

Weird dependency shenanigans: https://github.com/Islandora/islandora/pull/822/checks?check_run_id=1825556949#step:10:18

dannylamb · 2021-02-04T15:03:01Z

Can't seem to replicate using PHP 7.4 and Drupal 9.1.4

I'm noticing that we're on 9.1.1. Maybe i just need to get that to bump.

elizoller · 2021-02-10T20:00:33Z

So i did the following

spin up new playbook (fedora 5, php 7.4, drupal 9)
go into islandora module and check out no-gemini branch
go into crayfish and check out no-gemini branch
find all vendor directories with crayfish-commons and go into those and check out no-gemini branch
drush cr
make items ( i did so with the migrate_islandora_csv module because i didn't want to do it manually through the UI)

I didn't get any derivs for the migrated objects.
I suspect I'm missing some config changes somewhere?

for example the milliner log contains

[2021-02-10 13:54:21] app.DEBUG: > POST /milliner/media/field_media_image [] []
[2021-02-10 13:54:21] app.ERROR:  {"Exception":"[object] (GuzzleHttp\\Exception\\ConnectException(code: 0): cURL error 6: Could not resolve host: default (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) at /var/www/html/Crayfish/Milliner/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php:200)"} []
[2021-02-10 13:54:21] app.DEBUG: < 500 [] []

When I checked on a node page, i saw a fedora URI and i clicked it and it returned a 404, so i did index in fedora action and then clicked on the fedora URI again and it worked (no 404).

Media Fedora URIs appear to be working. but i didn't get any derivs for the migrated objects.

I will note that when i manually made an item node in the UI, it did persist to fedora without issue.
When i made a media manually through the UI, it had no issue with derivs.

elizoller · 2021-02-10T20:10:18Z

islandora.module

+        $media_source_service = \Drupal::service('islandora.media_source_service');
+        $source_file = $media_source_service->getSourceFile($entity);
+        $uri = $source_file->getFileUri();
+        $scheme = \Drupal::service('file_system')->uriScheme($uri);


This is deprecated in D9:
Replaced with \Drupal\Core\StreamWrapper\StreamWrapperManagerInterface::getScheme()

dannylamb · 2021-02-22T15:17:23Z

I'll try and follow your steps on a playbook and see what's up with derivatives.

dannylamb · 2021-02-25T19:13:53Z

Ok, after fixing that deprecation you found (which for me that class was straight up gone on 9.2.0-dev), I've got it working on a fresh playbook. No issues with derivatives that I can see.

Here's how I installed it (scraped from my bash history)

cd /var/www/html/drupal/web/modules/contrib/islandora
git fetch origin
git checkout no-gemini 
cd /var/www/html/drupal/vendor/islandora/crayfish-commons/
git fetch origin
git checkout no-gemini 
cd /var/www/html/Crayfish/
sudo git fetch origin
sudo git checkout no-gemini 
cd Milliner/vendor/islandora/crayfish-commons/
sudo git fetch origin
sudo git checkout no-gemini 
cd /var/www/html/drupal/
drush cr

Then I made an Image and uploaded an Original file and everything checked out. Everything makes its way into Fedora as expected without going through Gemini.

dannylamb · 2021-02-25T19:15:33Z

Ah... and now re-reading... Maybe the issue is just with migrate_islandora_csv. I'll give that a whirl.

dannylamb · 2021-02-25T19:38:57Z

@elizoller I bumped right into what you experienced. Turns out that I was calling the migration wrong. It needed both --userid=1 and --uri=localhost:8000 in order to work. After that it worked fine. By doing this I managed to test https://github.com/Islandora/migrate_islandora_csv/pull/40/files for ya. Works good!

Here's how I got and ran the code:

cd /var/www/html/drupal/web/modules/contrib
git clone https://github.com/asulibraries/migrate_islandora_csv.git
cd migrate_islandora_csv/
git checkout d9_updates
drush en migrate_islandora_csv
drush mim --userid=1 --uri=http://localhost:8000 --group migrate_islandora_csv

elizoller · 2021-02-25T19:47:12Z

did that fix all the derivative problems? if so, i'll re run and see what i get.

dannylamb · 2021-02-25T19:51:34Z

Derivatives worked out fine once I ran it with the right arguments.

elizoller · 2021-02-25T19:53:07Z

ok, i'll have another look

dannylamb · 2021-02-25T20:07:20Z

🙇

elizoller · 2021-02-26T15:12:18Z

@dannylamb i pulled in the new commit, rollback the migrations, re-imported them with the uri and userid flags and confirmed files, media, and nodes were all created. all are visible in drupal side, all expected derivatives were created, and fedora URI resolves correctly.
what else can i test for this?

dannylamb · 2021-03-01T16:54:16Z

I think that's about it as far as module code. I have ISLE rigged up to use this. I think the playbook will be mostly unaffected other than we'd want to remove the stuff to deploy gemini. Would you be willing to test if I throw up a quick PR to islandora-playbook?

elizoller · 2021-03-01T16:55:19Z

Definitely

dannylamb · 2021-03-01T16:55:36Z

🙇‍♂️

dannylamb · 2021-03-01T17:07:05Z

...and seeing now that all this stuff totally breaks the recast service 🎉

I don't think it's too bad since the service in Crayfish commons works both ways, but still. One more hurdle 🤕

elizoller · 2021-03-01T22:29:36Z

🤕

dannylamb · 2021-03-02T21:10:35Z

Ok, I've got recast working again. Fixing up its tests... all most there....

dannylamb · 2021-03-05T20:14:49Z

I just pushed up commits to Islandora/Crayfish#111 if you want to test again.

dannylamb · 2021-03-05T20:15:00Z

On to the playbook pull..

elizoller · 2021-03-12T18:37:09Z

@dannylamb any interested in #828 as a potential substitute for the sed line in this PR?

elizoller · 2021-03-12T18:38:25Z

@dannylamb any interest in #828 as a potential substitute for the sed line in this PR?

dannylamb · 2021-03-12T19:41:41Z

sed is gone 🔥

old habits die hard, i guess. didn't even look to the line above, which is a perfectly good example of setting a property using the command line 😅

elizoller

tested with Islandora-Devops/islandora-playbook#197

dannylamb added 2 commits January 8, 2021 18:07

Removing gemini

ce4d1fd

Forgot a test

94d4a73

dannylamb added 4 commits January 18, 2021 16:41

Merge branch '8.x-1.x' of https://github.com/Islandora/islandora into…

3c03101

… no-gemini

Update composer.json

2ad2e56

Adding views needed for reindexing

f4c4f5a

Merge branch 'no-gemini' of https://github.com/Islandora/islandora in…

73011fc

…to no-gemini

Trying to fix composer update fail on D9.1.1

d90200e

dannylamb added 2 commits February 4, 2021 11:31

Another attempt

77d2a01

Coding standards

90418ce

dannylamb marked this pull request as ready for review February 4, 2021 15:44

Removing some validation tests from IslandoraSettingsFormTest

13e6831

elizoller reviewed Feb 10, 2021

View reviewed changes

Update islandora.module

6709599

dannylamb added 2 commits March 12, 2021 15:32

Update composer.json

74976a9

Update build-8.x-1.x.yml

d6ceebd

elizoller approved these changes Mar 12, 2021

View reviewed changes

elizoller merged commit 1b6bea7 into 8.x-1.x Mar 12, 2021

noahwsmith mentioned this pull request May 4, 2021

Refactor needed post-removal of Gemini mjordan/islandora_premis#2

Closed

seth-shaw-unlv deleted the no-gemini branch September 1, 2021 20:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Removing gemini #822

Removing gemini #822

dannylamb commented Jan 8, 2021

elizoller commented Jan 11, 2021

dannylamb commented Feb 4, 2021

dannylamb commented Feb 4, 2021

elizoller commented Feb 10, 2021 •

edited

Loading

elizoller Feb 10, 2021

dannylamb commented Feb 22, 2021

dannylamb commented Feb 25, 2021

dannylamb commented Feb 25, 2021

dannylamb commented Feb 25, 2021

elizoller commented Feb 25, 2021

dannylamb commented Feb 25, 2021

elizoller commented Feb 25, 2021

dannylamb commented Feb 25, 2021

elizoller commented Feb 26, 2021

dannylamb commented Mar 1, 2021 •

edited

Loading

elizoller commented Mar 1, 2021

dannylamb commented Mar 1, 2021

dannylamb commented Mar 1, 2021

elizoller commented Mar 1, 2021

dannylamb commented Mar 2, 2021

dannylamb commented Mar 5, 2021

dannylamb commented Mar 5, 2021

elizoller commented Mar 12, 2021

elizoller commented Mar 12, 2021

dannylamb commented Mar 12, 2021

elizoller left a comment

Removing gemini #822

Removing gemini #822

Conversation

dannylamb commented Jan 8, 2021

What does this Pull Request do?

How should this be tested?

Interested parties

elizoller commented Jan 11, 2021

dannylamb commented Feb 4, 2021

dannylamb commented Feb 4, 2021

elizoller commented Feb 10, 2021 • edited Loading

elizoller Feb 10, 2021

Choose a reason for hiding this comment

dannylamb commented Feb 22, 2021

dannylamb commented Feb 25, 2021

dannylamb commented Feb 25, 2021

dannylamb commented Feb 25, 2021

elizoller commented Feb 25, 2021

dannylamb commented Feb 25, 2021

elizoller commented Feb 25, 2021

dannylamb commented Feb 25, 2021

elizoller commented Feb 26, 2021

dannylamb commented Mar 1, 2021 • edited Loading

elizoller commented Mar 1, 2021

dannylamb commented Mar 1, 2021

dannylamb commented Mar 1, 2021

elizoller commented Mar 1, 2021

dannylamb commented Mar 2, 2021

dannylamb commented Mar 5, 2021

dannylamb commented Mar 5, 2021

elizoller commented Mar 12, 2021

elizoller commented Mar 12, 2021

dannylamb commented Mar 12, 2021

elizoller left a comment

Choose a reason for hiding this comment

elizoller commented Feb 10, 2021 •

edited

Loading

dannylamb commented Mar 1, 2021 •

edited

Loading