not all schema collections suffix with _sets

[dwinston]

Description

Just like not all heroes wear capes, not all not all schema collections suffix with _sets.

Interpret perform_mongo_updates for non-id-having schema collections as simple insertions. Leave note in code about decision to insist on schema-supplied uniqueness signal.

This is a hack because e.g. https://w3id.org/nmdc/FunctionalAnnotationAggMember documents appear by eye to be unique via a compound key (metagenome_annotation_id, gene_function_id) and yet this is not explicit in the schema.

One potential solution is to auto-generate an id for such documents as a deterministic hash of the compound key, but any such id generation strategy should be unambiguous to a schema consumer such as nmdc-runtime.

For now, decision is to potentially re-insert "duplicate" documents, i.e. to interpret lack of id as lack of unique document identity for de-duplication.

Fixes #611

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

• test_perform_mongo_updates_functional_annotation_agg

Definition of Done (DoD) Checklist:

• My code follows the style guidelines of this project (have you run black nmdc_runtime/?)
• I have performed a self-review of my code
• I have added tests that prove my fix is effective or that my feature works, incl. considering downstream usage (e.g. https://github.com/microbiomedata/notebook_hackathons) if applicable.
• New and existing unit and functional tests pass locally with my changes (make up-test && make test-run)

[cmungall]

We can use https://w3id.org/linkml/unique_keys to declare a compound key (not totally understanding the broader issue though)

[shreddd]

I think we should review how this collection is typically generated based on:
https://github.com/microbiomedata/nmdc-aggregator/blob/main/generate_functional_agg.py

This will need some additional discussion. We can follow up on slack or in a small meeting.

[shreddd]

PS - this collection's role is to provide the aggregation tables for the data portal search (it will not be typically queried directly from the outside by end users), so we should keep that in mind.

One way to approach to this might be to model the set of mongo queries used in https://github.com/microbiomedata/nmdc-aggregator/blob/main/generate_functional_agg.py in the API.

[aclum]

@eecavanna @pkalita-lbl this PR is already closed, can we get a new ticket and PR to address the new issue

[eecavanna]

I'll create a new issue for it now. Thanks, @aclum.

Thanks, @pkalita-lbl, for reporting the issue and linking to the relevant commit.