Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update schema references to point to nmdc_materialized_patterns variant #887

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

eecavanna
Copy link
Collaborator

@eecavanna eecavanna commented Jan 30, 2025

On this branch, I updated some references to variants of the NMDC Schema other than the nmdc_materialized_patterns variant, so that they point to the nmdc_materialized_patterns variant.

Details

None.

Related issue(s)

Partially fixes #552

Related subsystem(s)

  • Runtime API (except the Minter)
  • Minter
  • Dagster
  • Project documentation (in the docs directory)
  • Translators (metadata ingest pipelines)
  • MongoDB migrations
  • Other

Testing

  • I tested these changes (explain below)
  • I did not test these changes

I will delegate the testing task to GitHub Actions.

Documentation

  • I have not checked for relevant documentation yet (e.g. in the docs directory)
  • I have updated all relevant documentation so it will remain accurate
  • Other (explain below)

Maintainability

  • Every Python function I defined includes a docstring (test functions are exempt from this)
  • Every Python function parameter I introduced includes a type hint (e.g. study_id: str)
  • All "to do" or "fix me" Python comments I added begin with either # TODO or # FIXME
  • I used black to format all the Python files I created/modified
  • The PR title is in the imperative mood (e.g. "Do X") and not the declarative mood (e.g. "Does X" or "Did X")

@eecavanna eecavanna linked an issue Jan 30, 2025 that may be closed by this pull request
@eecavanna eecavanna self-assigned this Jan 30, 2025
@eecavanna
Copy link
Collaborator Author

Hi @dwinston, you can add commits directly to this PR branch, in case you want to make changes related to "occurrence 1" without creating a separate PR.

@@ -32,9 +36,13 @@ def validate_json(data_path, schema_path, log_file):

def test_gold_study_json(
data_path="output/nmdc_etl/gold_study.json",
schema_path="../../../schema/nmdc.schema.json",
schema_path="/path/to/nmdc_materialized_patterns.schema.json",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we not doing this using the python package or linkml. @sierra-moxon please weigh in here but we can't just have a dummy path here

Copy link
Collaborator Author

@eecavanna eecavanna Feb 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't just have a dummy path here

Thanks—I agree. This was a mistake on my part.


Communicating which schema variant we expect, to the caller

The previous default value here would not work because this repository no longer contains a file at ../../../schema/nmdc.schema.json.

My intention by replacing the default value was to give a hint to the user about which variant of the schema we expect them to provide; i.e. the nmdc_materialized_patterns.schema.json variant, not the nmdc.schema.json variant. The code still won't work, though (neither file exists at the path specified).

I'll add a note to the (otherwise empty) docstring, saying that we expect the user to provide the path to the nmdc_materialized_patterns.schema.json variant of the schema when they invoke the function.


The default value

The Runtime repo no longer contains a schema file whose path we could use as the default value.


Invocations of this function

The only invocation of this function I see (in the repo) is the one right below the function's definition; i.e. this one:

if __name__ == "__main__":
    print("study test", test_gold_study_json())

That invocation relies on the function's default values.


The future of this script

Given that this script was already broken (due to the path being nonexistent) before this PR was created...

Do you have a sense for whether anyone uses this script anymore?

It was last updated in 2022 (before I updated the parameter's default value in this PR). I'm guessing someone wrote it to demonstrate something way back when.

Given that it lacked any documentation, maybe they did not expect it to be used by other people (or their future self) or maintained over time.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the latest commit (i.e. 2042c09), I removed all the obsolete, hard-coded file paths and replaced them with references to CLI arguments. This way, it is the user's responsibility to tell the script where it can find these files.

Copy link
Collaborator Author

@eecavanna eecavanna Feb 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd still like to remove this file from the repo if it's obsolete. That way, we and our successors don't spend time trying to salvage it as the rest of the code base evolves.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @turbomam what should we be using instead of the nmdc_materialized_patterns.schema.json

@aclum aclum requested a review from sierra-moxon February 1, 2025 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

nmdc-runtime should use materalized_pattern version of json schema
2 participants