Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CUMULUS-3859-1: Remove es from local api code and update active collection search #3861

Merged
merged 10 commits into from
Nov 20, 2024

Conversation

jennyhliu
Copy link
Contributor

@jennyhliu jennyhliu commented Nov 15, 2024

Summary: Summary of changes

Addresses [CUMULUS-3859: cumulus-dashboard: Update @cumulus packages for cypress tests

Changes

  • Removed ElasticSearch from local API server code
  • Updated CollectionSearch to filter additional granule fields for active collections

PR Checklist

  • Update CHANGELOG
  • Unit tests
  • Ad-hoc testing - Deploy changes and test manually
  • Integration tests

@jennyhliu jennyhliu marked this pull request as ready for review November 15, 2024 22:33
Copy link
Contributor

@npauzenga npauzenga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great @jennyhliu! I had a couple questions but mostly just looking for context.

@@ -79,7 +79,7 @@
"@babel/eslint-parser": "^7.24.1",
"@babel/preset-env": "^7.24.4",
"@docusaurus/eslint-plugin": "^2.3.0",
"@octokit/graphql": "2.1.1",
"@octokit/graphql": "^2.3.0",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This addresses GHSA-3xgq-45jj-v275 ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we have that added to the CL?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be added to master branch, whoever does that should add CL.

@@ -39,6 +48,13 @@ export class CollectionSearch extends BaseSearch {
super({ queryStringParameters }, 'collection');
this.active = (active === 'true');
this.includeStats = (includeStats === 'true');

// for active collection search, omit the fields which are for searching granules
if (this.active) {
Copy link
Contributor

@npauzenga npauzenga Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to be separate from CUMULUS-3859? Would you mind briefly explaining the use case here? We're searching for active collections and removing querystring params that correspond to granule properties instead of collections?

It's likely replacing some previous logic but I'm just not sure how it's used. Any context would be appreciated.

Copy link
Contributor Author

@jennyhliu jennyhliu Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the dashboard, user can filter active collection with the provider name which is granule field. Dashboard integration test is broken without the updates:
https://github.com/nasa/cumulus-dashboard/blob/v12.2.0/cypress/integration/collections_spec.js#L93-L94

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the fields exist in both granules and collections. With /collections/active endpoint, we want to filter these fields on granules table, not on collections table. The previous implementation overrides buildRangeQuery and applies range parameters on granules, and only filters collections with granule updated in the time frame.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think I understand. We have a set of queryparams on the /collections/active endpoint that correspond to granules, not collections. In this case we omit the granule params to fetch a collection.

It makes me wonder if there's a better way to structure this endpoint such that we don't send queryparams unrelated to the records it's returning but that's way out of scope and debatable.

Thanks for the clarification!

packages/db/src/search/StatsSearch.ts Show resolved Hide resolved
packages/db/src/translate/async_operations.ts Show resolved Hide resolved
@Jkovarik Jkovarik self-requested a review November 20, 2024 15:32
Copy link
Contributor

@npauzenga npauzenga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good @jennyhliu. I think given that this is a separate endpoint (/collections/active) it's ok that it's a bit complex and concerns both granules and collections.

@etcart was also reviewing so I'll approve as my questions are answered but don't want to step on toes.

@etcart
Copy link
Contributor

etcart commented Nov 20, 2024

awesome, and this has answered my concerns as well, thanks

@jennyhliu jennyhliu merged commit 0d7afd1 into feature/es-phase-2 Nov 20, 2024
3 checks passed
@jennyhliu jennyhliu deleted the jl/CUMULUS-3859/1 branch November 20, 2024 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants