Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for 'for' and 'until' in OAI-PHM ListRecords and ListIdentifiers verbs #63

Merged
merged 4 commits into from
Jan 13, 2023

Conversation

bmquinn
Copy link
Contributor

@bmquinn bmquinn commented Jan 11, 2023

Steps to test

  1. Index some data
  2. Run a ListRecords or ListIdentifiers request with either or both from and until parameters e.g. http://localhost:3000/oai?verb=ListRecords&metadataPrefix=oai_dc&from=2023-01-10T21:47:18.462902 (note the required format is YYYY-MM-DDThh:mm:ss:ffffff). Note the indexed_at values are inclusive of the date parameters.
  3. Tweak the values for from and until and you should see different results that fall within the dates.
  4. Test validation by passing in invalid date formats like 2023-01-10 or 2023-01-10T21:47:18.462902Z

@bmquinn bmquinn temporarily deployed to test January 12, 2023 17:45 — with GitHub Actions Inactive
@bmquinn bmquinn marked this pull request as ready for review January 12, 2023 17:51
@bmquinn bmquinn requested review from kdid and mbklein January 12, 2023 17:56
@mbklein
Copy link
Contributor

mbklein commented Jan 12, 2023

We're going to need to up the timeout on the OAI function in template.yaml because the function was timing out at 3 seconds on production data:

  oaiFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: handlers/oai.handler
      Description: Transforms works into OAI Records.
      Timeout: 60 # or whatever

@bmquinn bmquinn temporarily deployed to test January 12, 2023 20:00 — with GitHub Actions Inactive
@bmquinn bmquinn temporarily deployed to test January 12, 2023 20:04 — with GitHub Actions Inactive
Copy link
Contributor

@mbklein mbklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving but also wondering whether it would be more useful to harvesters for us to use modified_date to determine freshness instead of create_date. I can't really remember how harvesters work in practice, so if it's only discovery of new records they care about, this is definitely what's called for.

@bmquinn
Copy link
Contributor Author

bmquinn commented Jan 12, 2023

Approving but also wondering whether it would be more useful to harvesters for us to use modified_date to determine freshness instead of create_date. I can't really remember how harvesters work in practice, so if it's only discovery of new records they care about, this is definitely what's called for.

I think we actually need both here, create_date in order to determine the earliestDatestamp element's value in Identify requests and modified_date in from and until in search verbs.

@bmquinn bmquinn temporarily deployed to test January 12, 2023 22:20 — with GitHub Actions Inactive
@mbklein mbklein merged commit 95381c1 into deploy/staging Jan 13, 2023
@mbklein mbklein deleted the 3399-oai-from branch January 13, 2023 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants