Generate ARK URLs for resources #1161

benjamingeer · 2019-01-14T15:11:27Z

In API v2, return an ARK URL as part of the metadata of each resource (in ReadResourceV2.toJsonLD).
Add documentation:
- docs/src/paradox/03-apis/api-v2/resource-permalinks.md
- docs/src/paradox/05-internals/design/api-v2/ark.md
Change all resources in test data to use Knora's standard resource IRI format, so we can generate ARK URLs for them.

Resolves #844.

…of 0.

…s and ARK URLs.

- Don't use a delimiter before the check digit. - Use a slash as the delimiter before the timestamp.

- Make version, project ID, and resource ID an object hierarchy. - Make the timestamp an object variant. - Escape - as =.

- Don't use hyphens or colons in timestamps in ARK URLs.

…ing). - Use 0-based character values in check digits, otherwise the maths don't work. - Add missing project IDs to IRIs in test data. - Replace remaining occurrences of data.knora.org with rdfh.ch.

tobiasschweizer · 2019-01-18T07:33:07Z

@benjamingeer I have data that still has http://data.knora.org instead of http://rdfh.ch

This is because we have a productive Knora as of summer 2016. The IRIs are referred from XML transcriptions.

benjamingeer · 2019-01-18T09:06:57Z

@tobiasschweizer We can fix all your IRIs together next week, it's not difficult, I've just done this for the Incunabula test data.

tobiasschweizer · 2019-01-18T09:12:03Z

@benjamingeer I also think that it won't be hard. I actually used emacs regex search and replace to adapt the permissions.

I just have to do the same thing in the XMLs where the IRIs are used.

@subotic Maybe the internal beol server should be re-installed from scratch. This time, I would like to use docker. Could you give me a hand?

- Update docs and release notes.

subotic · 2019-01-18T14:23:06Z

@subotic Maybe the internal beol server should be re-installed from scratch. This time, I would like to use docker. Could you give me a hand?

Sure, we can take the deployment configuration from production and change it for the internal beol server.

# Conflicts: # webapi/src/test/scala/org/knora/webapi/responders/admin/PermissionsResponderADMSpec.scala

# Conflicts: # docs/src/paradox/00-release-notes/next.md # salsah1/src/typescript_interfaces/sampleRequests/sampleResourceResponses.ts # webapi/src/test/scala/org/knora/webapi/responders/v1/ResourcesResponderV1Spec.scala # webapi/src/test/scala/org/knora/webapi/responders/v1/SearchResponderV1Spec.scala

…stamps.

subotic

Can we please move ark.py to its own repository?

webapi/src/test/scala/org/knora/webapi/e2e/v2/SearchRouteV2R2RSpec.scala

benjamingeer · 2019-01-21T10:18:54Z

Can we please move ark.py to its own repository?

Yes, good idea.

subotic · 2019-01-21T10:19:41Z

I've taken the liberty and moved it into its own repository :-)

https://github.com/dhlab-basel/ark-resolver/

benjamingeer · 2019-01-21T10:30:26Z

Great, thank you!

subotic · 2019-01-21T11:12:49Z

Thanks :-)

subotic · 2019-01-21T12:56:50Z

@benjamingeer Any particular reason why the resolver is written in python and not in scala? You know that you are now the sole responsible person for maintaining the code ;-)

- Refactor ARK design and API docs. - Fix broken test.

benjamingeer · 2019-01-21T14:31:19Z

Any particular reason why the resolver is written in python and not in scala?

Two reasons:

It's such a simple program, I think it doesn't need a JVM consuming gigabytes of memory.
I think some people might want to write their own ARK resolvers for use with Knora, they'll want to look at this one to see how it works, and it might be easier for them to read code in Python than in Scala.

you are now the sole responsible person for maintaining the code ;-)

That's OK, I can live with maintaining this, because it hardly does anything. :)

ezbinden · 2019-01-24T13:37:28Z

Hi Ben, Ivan
Could this be a breaking change for some of our partners? If yes they should be notified via the DaSCH team list
Cheers, E.

feature (webapi): Generate ARK URLs for resource IRIs (ongoing).

889f6dc

benjamingeer mentioned this pull request Jan 14, 2019

Ben's PR history #571

Open

Benjamin Geer added 16 commits January 14, 2019 17:07

fix (webapi): When calculating checksum, don't generate a char value …

9497e4c

…of 0.

feature (webapi): Add version number to ARK URL.

50383c6

Merge branch 'develop' into wip/844-ark-urls

82451ec

feature (ark): Add Python utility for converting between resource IRI…

cc87a75

…s and ARK URLs.

feature (ark): Fix typo.

a94b122

feature (ark): Add copyright notice.

e8d3f18

feature (ark): Tweak delimiters.

5406a91

- Don't use a delimiter before the check digit. - Use a slash as the delimiter before the timestamp.

style (ark): Clarify code and comments.

538e443

fix (ark): Conform better to ARK spec.

f10837e

- Make version, project ID, and resource ID an object hierarchy. - Make the timestamp an object variant. - Escape - as =.

Merge branch 'develop' into wip/844-ark-urls

2e94acd

Merge branch 'develop' into wip/844-ark-urls

c7575ff

Merge branch 'develop' into wip/844-ark-urls

3973f8a

Merge branch 'develop' into wip/844-ark-urls

829b1d5

feature (ark): Start refactoring Python code to make a server.

6aedbd6

- Don't use hyphens or colons in timestamps in ARK URLs.

feature (ark): Add property arkUrl to knora-api ontologies.

bef79ab

feature (ark): Return ARK URLs with resource metadata in API v2 (ongo…

ac0a3df

…ing). - Use 0-based character values in check digits, otherwise the maths don't work. - Add missing project IDs to IRIs in test data. - Replace remaining occurrences of data.knora.org with rdfh.ch.

benjamingeer changed the title ~~Generate and validate ARK URLs for resource IRIs~~ Generate and resolve ARK URLs for resources Jan 16, 2019

Benjamin Geer added 8 commits January 17, 2019 12:20

feature (ark): Fix more test data and tests.

09885a2

feature (ark): Fix integration test data.

c23fc0a

feature (ark): Fix more tests.

9e62803

feature (ark): Fix more tests.

ce356d1

feature (ark): Fix test again.

46456b4

feature (ark): Add ARK redirect server.

f0ee2a0

feature (ark): Add comments and clarify config setting names.

7e00204

docs (ark): Document ARK URLs.

515cd2c

benjamingeer mentioned this pull request Jan 17, 2019

Define ARK IDs #844

Closed

docs (ark): Add blank line to list.

8b5591c

feature (ark): Don't use dots in timestamps.

124f622

- Update docs and release notes.

Benjamin Geer added 5 commits January 18, 2019 15:27

Merge branch 'develop' into wip/844-ark-urls

1caac2c

# Conflicts: # webapi/src/test/scala/org/knora/webapi/responders/admin/PermissionsResponderADMSpec.scala

Merge branch 'develop' into wip/844-ark-urls

dc734f6

refactor (ark): Move ark directory to top level.

71e0ce4

feature (ark): Plan to return separate ARK URLs with and without time…

6c3bf7b

…stamps.

benjamingeer requested a review from subotic January 20, 2019 08:56

docs (ark): Start adding design doc about ARK URLs.

de5522c

subotic requested changes Jan 21, 2019

View reviewed changes

subotic reviewed Jan 21, 2019

View reviewed changes

webapi/src/test/scala/org/knora/webapi/e2e/v2/SearchRouteV2R2RSpec.scala Outdated Show resolved Hide resolved

subotic requested changes Jan 21, 2019

View reviewed changes

webapi/src/test/scala/org/knora/webapi/e2e/v2/SearchRouteV2R2RSpec.scala Outdated Show resolved Hide resolved

docs (ark): Add ARK URL design doc.

d604a94

Benjamin Geer added 2 commits January 21, 2019 11:21

test (search): Set test-data-generation flag to false.

43a561b

docs (ark): Refer to ARK resolver in separate repository.

f398137

benjamingeer changed the title ~~Generate and resolve ARK URLs for resources~~ Generate ARK URLs for resources Jan 21, 2019

subotic approved these changes Jan 21, 2019

View reviewed changes

docs (ark): Fix broken link.

64b2411

feature (ark): Remove ark.py now that it has its own repository.

44d5408

- Refactor ARK design and API docs. - Fix broken test.

benjamingeer merged commit 2b92c7d into develop Jan 21, 2019

benjamingeer deleted the wip/844-ark-urls branch January 21, 2019 15:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate ARK URLs for resources #1161

Generate ARK URLs for resources #1161

benjamingeer commented Jan 14, 2019 •

edited

Loading

tobiasschweizer commented Jan 18, 2019 •

edited

Loading

benjamingeer commented Jan 18, 2019

tobiasschweizer commented Jan 18, 2019

subotic commented Jan 18, 2019

subotic left a comment

benjamingeer commented Jan 21, 2019

subotic commented Jan 21, 2019

benjamingeer commented Jan 21, 2019

subotic commented Jan 21, 2019

subotic commented Jan 21, 2019

benjamingeer commented Jan 21, 2019

ezbinden commented Jan 24, 2019

Generate ARK URLs for resources #1161

Generate ARK URLs for resources #1161

Conversation

benjamingeer commented Jan 14, 2019 • edited Loading

tobiasschweizer commented Jan 18, 2019 • edited Loading

benjamingeer commented Jan 18, 2019

tobiasschweizer commented Jan 18, 2019

subotic commented Jan 18, 2019

subotic left a comment

Choose a reason for hiding this comment

benjamingeer commented Jan 21, 2019

subotic commented Jan 21, 2019

benjamingeer commented Jan 21, 2019

subotic commented Jan 21, 2019

subotic commented Jan 21, 2019

benjamingeer commented Jan 21, 2019

ezbinden commented Jan 24, 2019

benjamingeer commented Jan 14, 2019 •

edited

Loading

tobiasschweizer commented Jan 18, 2019 •

edited

Loading