fix: fix processing logs #3922
Merged
fix: fix processing logs #3922
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## main #3922 +/- ##
==========================================
+ Coverage 65.16% 65.19% +0.03%
==========================================
Files 49 49
Lines 3425 3428 +3
==========================================
+ Hits 2232 2235 +3
Misses 1193 1193
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
nayib-jose-gloria
approved these changes
Jan 11, 2023
nayib-jose-gloria
added a commit
that referenced
this pull request
Jan 23, 2023
* feat(MUIv5): Upgrade SDS and MUI (#3734) * feat(MUIv5): Upgrade SDS and MUI * update ButtonIcon * fix Filter * upgrade sds * wip fix highlight * replace Blueprint HTMLTable * fix QuickSelect style * remove lint ignore * fix QuickSelect style * fix: Fix the re-tagging issue due to invalid registryId (#3781) Fixes the following error in Docker re-tag: ``` [ERROR]: error getting Image: operation error ECR: BatchGetImage, https response error StatusCode: 400, RequestID: f00322e8-7a33-4e28-a07f-93f2bdaa25bb, InvalidParameterException: Invalid parameter at 'registryId' failed to satisfy constraint: 'must satisfy regular expression [0-9]{12}' ``` * feat: portal redesign (#3485) * prep * more stuff * missing line * changes * bunch of changes * interface and tests * Update backend/layers/common/entities.py Co-authored-by: Andrew Tolopko <[email protected]> * temp stuff * refactor, add documentation * more refactor * temp * typo * Update backend/layers/persistence/persistence.py Co-authored-by: Andrew Tolopko <[email protected]> * more tests * Update backend/layers/persistence/persistence.py Co-authored-by: Andrew Tolopko <[email protected]> * Update backend/layers/persistence/persistence.py Co-authored-by: Andrew Tolopko <[email protected]> * more tests * refactor * Update backend/layers/persistence/persistence.py Co-authored-by: Andrew Tolopko <[email protected]> * more tests * even more stuff * Changes * PR comments * PR suggestions, as code changes * changes * Comments * Remove owner from collection metadata * rename * more work * Remove all authorization from the business logic layer * Assertions for metadata validation errorS * More and more stuff * convert everything to id classes * First pass * more files * more stuff * TestCreateCollection ✅ * TestGetCollectionVersion ✅ * TestGetAllCollections ✅ * Add one more test * chore: business layer tests for update collection (#3524) * chore: business layer tests for update collection * Update tests/unit/backend/layers/business/test_business.py Co-authored-by: Daniel Hegeman <[email protected]> * remove old test Co-authored-by: Daniel Hegeman <[email protected]> * signature * Overhaul dataset processing status * missing files * fix enum mixin * comments * few fixes * TestUpdateCollectionDatasets ✅ * TestGetDataset ✅ * TestUpdateDataset ✅ * TestCollectionOperations ✅ * Link validation * Move business interface to another file * initial dump * missing files * more tests * stuff * missing files * more stuff * chore: add DatasetMetadata missing fields (#3572) * changes * Add name, x_approximate_distribution * baby steps * test__get_collection__ok ✅ * many more tests ✅ * small assertion * Link names can be optional * stuff * fix bug + enforce immutability in several places * many more tests ✅ * moving to datasets * chore: deepcopy in persistence_mock (#3578) * deepcopy in persistence_mock * Added comment * Add DatasetArtifactId * more tests * start * functions * baby steps * ingest_dataset now also returns canonical dataset_id * test__get_all_datasets_for_index_with_ontology_expansion ✅ * missing files * test__get_dataset_assets ✅ * rework base class * revision tests - first pass * stuff * TestRevision ✅ * TestDeleteRevision ✅ * Fix a bunch of pending tests * chore: add dataset_version -> collection_id link * temp * chore: add methods to get versions from a canonical collection_id * one more test * upload tests * more tests ✅ * upload link tests ✅ * Publish collection ✅ * chore: add dataset_version -> collection_id link (#3585) * chore: collection versions can only be created one at a time (#3582) * chore: collection versions can only be created one at a time * add a case for non existing collections * chore: add methods to get versions from a canonical collection_id (#3586) * last minute changes * initial stuff * more stuff * several advancements * stuff * more stuff * fix: Migration task fails to start (#3588) * Fix commits * stuff * stuff * temp change * unit tests attempt * more unit tests * more unit tests * happy path * s3 provider * more implementations * stuff * Changes * add mock * stuff * typo * typo * meta endpoint * conflict * remove outdated stuff * Remove old test * one more test * fixes * stuff * tmp remove curator_name * Add field * one fix * more stuff * overhaul * Rename class * rename published_at -> originally_published_at * feat: explicit canonical collection (#3622) * stuff * tmp remove curator_name * Add field * one fix * more stuff * overhaul * Rename class * rename published_at -> originally_published_at * timestamps * timestamps 2 * feat: persistence layer Signed-off-by: nayib-jose-gloria <[email protected]> * feat: add persistence implementation (first draft) Signed-off-by: nayib-jose-gloria <[email protected]> * feat: add finalization of canonical dataset on collection version publish Signed-off-by: nayib-jose-gloria <[email protected]> * feat: redesign canonical dataset (#3627) * feat: canonical dataset * published_at in API * fix tests * partial changes * add set-up to create schema from persistence orm Signed-off-by: nayib-jose-gloria <[email protected]> * fixes to the ORM * fix method call * fix: use DatabaseProvider for test_business and update persistence layer accordingly Signed-off-by: nayib-jose-gloria <[email protected]> * stuff * stuff * fix TestCollectionOperations tests Signed-off-by: nayib-jose-gloria <[email protected]> * persistence layer changes (#3650) * checkpoint * more stuff * one more test * stuff and more stuff * final * tests * changes * pr feedback Signed-off-by: nayib-jose-gloria <[email protected]> * Various bugs&fixes * fixes Signed-off-by: nayib-jose-gloria <[email protected]> * fix tests and pr feedback Signed-off-by: nayib-jose-gloria <[email protected]> * feat: processing layer (#3603) * Fixes for API tests * get_collection_index * Missing files * bunch of changes * properly set revision_of * failure * failure layer * more merge * feat: migration script + some fixes (#3681) * first pass * second pass * stuff * stuff * more stuff * bunch of changes * script * parametrize db name * docs * more debug stuff * remove debug stuff * black and flake8 Signed-off-by: Trent Smith <[email protected]> * fix: publisher metadata correct type (#3699) * fix: publisher metadata correct type * fix: publisher metadata correct type * typo * fix: allows business tests to be run by both persistence mock and as an integration test with postgres docker container Signed-off-by: nayib-jose-gloria <[email protected]> * lint Signed-off-by: nayib-jose-gloria <[email protected]> * feat: revise at for curation api (#3675) Add revised_at and validation message to backend.layers.common.entity - change UserInfo.user_id to a property. Simplifies accessing the variable. - use pydantic.datalclass. This gives us type checking for free. - Moved errors in BusinessException to be an instance variable. Making it is class variable will have unintended consequences. - Fixed pointer in persistence_mock.py - Populating revised_at in persistence_mock.py * fix merge conflict Signed-off-by: Trent Smith <[email protected]> * fix: get portal API tests working with persistence mock Signed-off-by: nayib-jose-gloria <[email protected]> * fix: get portal API tests working with database provider implementation Signed-off-by: nayib-jose-gloria <[email protected]> * refactor: simplify session management + include option to run tests as unit tests or integration tests Signed-off-by: nayib-jose-gloria <[email protected]> * pr feedback Signed-off-by: nayib-jose-gloria <[email protected]> * fix: processing tests (#3715) * fix: strip -> removesuffix (#3716) * chore: redesign run tests in GHA (#3694) * chore: redesign run tests in GHA * pip * switch to docker * remove import * all tests * Lint * overhaul test commands * fix workflow * feat: add canonical collection tombstoning to portal redesign (#3740) * linting Signed-off-by: nayib-jose-gloria <[email protected]> * feat: support canonical collection tombstoning Signed-off-by: nayib-jose-gloria <[email protected]> * linting + pr feedback Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> * feat: set validation message (#3735) * feat: set validation message * Lint * chore: redesign miscellaneous fixes (#3739) * stuff * lint * fix test * add published_at to ignored fields for revision diff * add crossref provider * PR comments & lint * move explorer_url * missing files * chore: restore processing tests (#3736) * not quite there yet * stuff * missing files * fix * lint * chore: restore cloudfront invalidation (#3761) * chore: restore cloudfront invalidation * rename * rename file * missing files * chore: curator name (#3780) * feat: translate curation api for redesign (#3690) * fix typo * resolve conflicts Signed-off-by: Trent Smith <[email protected]> * curation api WIP Signed-off-by: Trent Smith <[email protected]> * curation_api Signed-off-by: Trent Smith <[email protected]> * curation_api Signed-off-by: Trent Smith <[email protected]> * consolidate tests Signed-off-by: Trent Smith <[email protected]> * resolve DOIs Signed-off-by: Trent Smith <[email protected]> * Fix s3_upload_credentials.py Signed-off-by: Trent Smith <[email protected]> * Fix some get collection id tests Signed-off-by: Trent Smith <[email protected]> * Fix post Revisions Fix some exiting tests after merging from target branch. Signed-off-by: Trent Smith <[email protected]> * Fix Get Collections public test Signed-off-by: Trent Smith <[email protected]> * Fix get collections tests Signed-off-by: Trent Smith <[email protected]> * Fix get collection tests Signed-off-by: Trent Smith <[email protected]> * Fix get collection tests Signed-off-by: Trent Smith <[email protected]> * feat: curation API for datasets (#3708) * init * stuff * baby steps * new stuff * end * more stuff * Remove ApiCommon * Update backend/portal/api/curation/v1/curation/collections/collection_id/datasets/dataset_id/actions.py Co-authored-by: Trent Smith <[email protected]> * Update backend/portal/api/curation/v1/curation/collections/collection_id/datasets/dataset_id/actions.py Co-authored-by: Trent Smith <[email protected]> * PR comments Co-authored-by: Trent Smith <[email protected]> * fixing patch tests Signed-off-by: Trent Smith <[email protected]> * consolidating tests Signed-off-by: Trent Smith <[email protected]> * fixing get portal api index Signed-off-by: Trent Smith <[email protected]> * Update tests/unit/backend/layers/business/test_business.py * Update backend/layers/business/business.py * fix typos Signed-off-by: Trent Smith <[email protected]> Signed-off-by: Trent Smith <[email protected]> Co-authored-by: Emanuele Bezzi <[email protected]> * chore: curator API fix tests (#3782) * fix typo * resolve conflicts Signed-off-by: Trent Smith <[email protected]> * curation api WIP Signed-off-by: Trent Smith <[email protected]> * curation_api Signed-off-by: Trent Smith <[email protected]> * curation_api Signed-off-by: Trent Smith <[email protected]> * consolidate tests Signed-off-by: Trent Smith <[email protected]> * resolve DOIs Signed-off-by: Trent Smith <[email protected]> * Fix s3_upload_credentials.py Signed-off-by: Trent Smith <[email protected]> * Fix some get collection id tests Signed-off-by: Trent Smith <[email protected]> * Fix post Revisions Fix some exiting tests after merging from target branch. Signed-off-by: Trent Smith <[email protected]> * Fix Get Collections public test Signed-off-by: Trent Smith <[email protected]> * Fix get collections tests Signed-off-by: Trent Smith <[email protected]> * Fix get collection tests Signed-off-by: Trent Smith <[email protected]> * Fix get collection tests Signed-off-by: Trent Smith <[email protected]> * feat: curation API for datasets (#3708) * init * stuff * baby steps * new stuff * end * more stuff * Remove ApiCommon * Update backend/portal/api/curation/v1/curation/collections/collection_id/datasets/dataset_id/actions.py Co-authored-by: Trent Smith <[email protected]> * Update backend/portal/api/curation/v1/curation/collections/collection_id/datasets/dataset_id/actions.py Co-authored-by: Trent Smith <[email protected]> * PR comments Co-authored-by: Trent Smith <[email protected]> * fixing patch tests Signed-off-by: Trent Smith <[email protected]> * consolidating tests Signed-off-by: Trent Smith <[email protected]> * fixing get portal api index Signed-off-by: Trent Smith <[email protected]> * Update tests/unit/backend/layers/business/test_business.py * Update backend/layers/business/business.py * fix typos Signed-off-by: Trent Smith <[email protected]> * init * rollback * checkpoint * checkpoint * checkpoint * checkpoint * more tests * Asset tests * more test fixes * more test fixes * comment * Linter * fix * linter * fixes * improve tests * tests * remove monkeypatch * Update backend/portal/api/curation/v1/curation/collections/actions.py Co-authored-by: Trent Smith <[email protected]> Signed-off-by: Trent Smith <[email protected]> Co-authored-by: Trent Smith <[email protected]> Co-authored-by: Trent Smith <[email protected]> * chore: add curator_name to migration script (#3791) * chore: redesign last minute fixes (#3792) * fixes * more fixes * lint * feat: add integration test mode as a gha test step + makefile command (#3788) * feat: add integration test mode as a gha test step + makefile command Signed-off-by: nayib-jose-gloria <[email protected]> * pin dataclasses-json version in requirements + reintroduce step to rebuild backend container on changed files/dockerfiles Signed-off-by: nayib-jose-gloria <[email protected]> * add pydantic to processing requirements, rebuild containers with no cache Signed-off-by: nayib-jose-gloria <[email protected]> * remove --no-cache flag from rebuild commands Signed-off-by: nayib-jose-gloria <[email protected]> * remove pydantic to pass tests Signed-off-by: nayib-jose-gloria <[email protected]> * re-add dataclasses import from std lib Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> Co-authored-by: Emanuele Bezzi <[email protected]> * chore: last minute fixes #2 (#3794) * update ontology test Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: Trent Smith <[email protected]> Co-authored-by: Andrew Tolopko <[email protected]> Co-authored-by: Andrew Tolopko <[email protected]> Co-authored-by: Daniel Hegeman <[email protected]> Co-authored-by: Alex Lokshin <[email protected]> Co-authored-by: nayib-jose-gloria <[email protected]> Co-authored-by: Trent Smith <[email protected]> Co-authored-by: Nayib Gloria <[email protected]> Co-authored-by: Trent Smith <[email protected]> * fix: missing 'e' in explorer_url + dataset id in meta endpoint (#3795) * fix: missing 'e' in explorer_url * more * feat: dynamic sitemap and robots.txt file (#3753) Co-authored-by: Timmy Huang <[email protected]> * fix: version_id in meta endpoint (#3809) * fix: version_id in meta endpoint * fix tests * fix: support for collection partial updates (#3812) * fix: support for collection partial updates * Lint * fix e2e after MUIv5 upgrade (#3813) Co-authored-by: atarashansky <[email protected]> * fix: y-axis scrolling jump on heatmap download (#3691) * fix: y-axis scrolling jump on heatmap download * restoring scrollTop position after create image and before download * feat: adding tint screen on download load * fix: functional tests portal redesign compatibility (#3815) * fix: failing functional tests related to portal redesign Signed-off-by: nayib-jose-gloria <[email protected]> * fix: failing functional tests related to portal redesign Signed-off-by: nayib-jose-gloria <[email protected]> * fix: lint Signed-off-by: nayib-jose-gloria <[email protected]> * fix: grab id from DatasetId Signed-off-by: nayib-jose-gloria <[email protected]> * fix: rollback container-functionaltest change Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> * test(wmg): Fix Source Data (#3816) * test(wmg): Fix Data Source part 2 (#3817) * fix: only add original_ids on unpublished revisions of published datasets (#3820) * fix: only add original_ids on unpublished revisions of published datasets Signed-off-by: nayib-jose-gloria <[email protected]> * fix: fix and add portal API tests + update test method return type hints Signed-off-by: nayib-jose-gloria <[email protected]> * lint Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> * fix: bad CrossrefProvider imports + reintroduce auth0 retry (#3819) * fix: bad CrossrefProvider imports * fix tests * lint * last minute fix * fix: for portal API dataset status and delete endpoints, fetch latest collection version (#3822) Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> * chore: fix test snapshot (#3818) Co-authored-by: atarashansky <[email protected]> Co-authored-by: Daniel Hegeman <[email protected]> * chore: support INITIALIZED dataset status (#3824) * chore: support INITIALIZED dataset status * try to fix tests * feat: PATCH collection curator API redesign (#3814) * init * more tests * end * PR comments * fix import * fix: remove original_id and fix post-deploy tests (#3826) Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> * fix: test_api_key_crud functional test (#3828) * fix: remove original_id and fix post-deploy tests Signed-off-by: nayib-jose-gloria <[email protected]> * fix: test_api_key_crud functional test Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> * chore: redesign dataset submissions lambda (#3827) * init * more * fix tests * lint * mock config * Bunch of fixes * more changes * Missing dependencies * fix: redesign submissions lambda fix 1 (#3832) * fix: redesign submissions lambda fix 1 * lint * fix: redesign submissions lambda fix 2 (#3833) * fix: redesign submissions lambda fix 2 * missing providers * chore: double lookup for GET dataset in curation API (#3834) * chore: double lookup for GET dataset in curation API * linter * add comment * fix: send slack alerts on failing post-deploy tests (#3821) * fix: send slack alerts on failing post-deploy tests Signed-off-by: nayib-jose-gloria <[email protected]> * fix: use workflow status GHA action step to check overall workflow status at the end of workflow Signed-off-by: nayib-jose-gloria <[email protected]> * fix workflow overall failure check Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> * feat: marker genes unavailable for cell types <25 (#3825) * feat: marker genes unavailable for cell types <25 Co-authored-by: atarashansky <[email protected]> * chore: reintroduce wmg unit tests + migrate to redesign (#3837) * chore: reintroduce wmg unit tests to a separate action * step 1: fix get_dataset_s3_uris * add layers to dockerfile * 2nd endpoint * remove raw h5ad * chore: add filter for local.h5ad to wmg extract function (#3843) * chore: reintroduce wmg unit tests to a separate action * rename * step 1: fix get_dataset_s3_uris * add layers to dockerfile * dep * Fix * checkpoint * test #1 * 2nd endpoint * lint * remove raw h5ad * fix * fix: add local.h5ad trailing * Linter * fix(fmg): fix ordering and remove p-value (#3831) * fix: mainting top-n ordering for marker genes * accept new response in frontend * remove p-value from chart * add back p-value to api yml * fmt + lint * update tests * p-value back in example * fix remaining tests * blacked and fixed unit tests * blacked unit tests * fixed formatting Co-authored-by: atarashansky <[email protected]> Co-authored-by: Seve Badajoz <[email protected]> Co-authored-by: atarashansky <[email protected]> Co-authored-by: atarashansky <[email protected]> * feat: Exclude Blood from FMG (#3839) * feat: Exclude Blood from FMG * change effect size to marker score (#3855) Co-authored-by: atarashansky <[email protected]> * chore(deps): bump json5 from 1.0.1 to 1.0.2 in /frontend (#3842) Bumps [json5](https://github.com/json5/json5) from 1.0.1 to 1.0.2. - [Release notes](https://github.com/json5/json5/releases) - [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md) - [Commits](json5/json5@v1.0.1...v1.0.2) --- updated-dependencies: - dependency-name: json5 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Timmy Huang <[email protected]> * docs: add discover api data model img for cell census schema docs reference (#3844) * chore: RAW_H5AD type applied in migration + processing (#3862) * chore: RAW_H5AD type applied in migration + processing * Lint Co-authored-by: Nayib Gloria <[email protected]> * feat(WMG+FMG): improve GE layout, rebuild y axis chart, and fix inconsistent gene spacing (#3811) * redo GE layout and rebuild y axis chart * remove vestigial variables * update * checkpoint * update Co-authored-by: atarashansky <[email protected]> Co-authored-by: atarashansky <[email protected]> * update tests (#3868) Co-authored-by: atarashansky <[email protected]> * fix(portal redesign): handle raw h5ads at api layer in portal and curation API (#3873) * fix(portal redesign): handle raw h5ads at api layer in portal and curation API Signed-off-by: nayib-jose-gloria <[email protected]> * remove unused import Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> * fix(wmg): don't send requests if only genes or tissues have been removed (#3859) * fix(wmg): don't send requests if only genes or tissues have been removed * lint Co-authored-by: Seve Badajoz <[email protected]> * feat(fmg): shift marker genes to front if already selected when adding (#3874) * branch init * feat(fmg): shift marker genes to front if already selected when adding Co-authored-by: Seve Badajoz <[email protected]> * fix: downloaded image being cut off (#3869) * fix(wmg): y axis is now properly sticky (#3877) * fix y axis stickiness when scrolling horizontally * add comment * remove inline container Co-authored-by: atarashansky <[email protected]> * fix(wmg): image download (#3880) * chore(wmg): rewrite x axis using react (#3879) * rewrite x axis as react * update * update * prettier * remove dup prop Co-authored-by: atarashansky <[email protected]> * fix(WMG): remove step 2 when genes are added and heatmap is loading (#3763) * branch init * remove isLoading check for gene step removal Co-authored-by: atarashansky <[email protected]> * fix(wmg): Select Tissues/Genes dropdown not triggering onClick event (#3881) * chore(fmg): fmg documentation (#3870) * fmg documentation * update deps * update deps Co-authored-by: atarashansky <[email protected]> * feat: FMG tooltip doc link (#3852) * feat: FMG tooltip doc link * adding doc link * Update routes.ts Co-authored-by: atarashansky <[email protected]> * fix(wmg): echarts init (#3884) * change link (#3887) Co-authored-by: atarashansky <[email protected]> * fix e2e after x axis rewrite (#3888) Co-authored-by: atarashansky <[email protected]> * chore: only return dataset_deployments on CXG completion (#3851) * chore: only return dataset_deployments on CXG completion * fix test + lint * empty list instead of None * chore: translates the portal API module from class-based to function-based (#3878) * init * chore: migrate portal API to be function based instead of class based * linter * rename router.py -> providers.py * Add missing files * chore: update alembic migration to use new database (#3841) - Upgrade SQLAlchemy for the processing container so it matches the other containers - Upgrade Alembic for local development. - factor out "persistance_schema" into a constant to avoid naming issues. - Update alembic to support both the new and legacy ORM for now. The legacy ORM can be removed in a future migration. - Add an Alembic migration for the redesign database changes. This is will not affect the legacy database. - Modify `migrate_redesign_write` so it no longer create the new database tables. This should be handled by the automated migration processes in GHA. - Fixe the legacy ORM to match the actual database. - Update `create_db` to use the new ORM. - updated sqlalchemy in the processing container to be current with other containers. - Add Make recipe `db/check` to make it easier to detect if the ORM is different from the database. - Add a try..except block in `DatabaseProvider._drop_schema` so prevent breaking if the schema does not exist. - Add support for creating both the legacy and new database. - removing CORPORA_LOCAL_DEV from setup_dev_data.sh. * fix(wmg): still check for equality on non-checked filters when checking to clobber queries (#3898) Co-authored-by: Seve Badajoz <[email protected]> * update (#3886) Co-authored-by: atarashansky <[email protected]> * update (#3901) Co-authored-by: atarashansky <[email protected]> * chore: update marker gene unit test (#3902) * update test * empty Co-authored-by: atarashansky <[email protected]> * update (#3885) Co-authored-by: atarashansky <[email protected]> * refactor: persistence layer clean-up (#3835) * refactor: persistence layer clean-up Signed-off-by: nayib-jose-gloria <[email protected]> * lint Signed-off-by: nayib-jose-gloria <[email protected]> * update missing id changes Signed-off-by: nayib-jose-gloria <[email protected]> * lint fix Signed-off-by: nayib-jose-gloria <[email protected]> * more missing id's Signed-off-by: nayib-jose-gloria <[email protected]> * remove unused import Signed-off-by: nayib-jose-gloria <[email protected]> * correct id name Signed-off-by: nayib-jose-gloria <[email protected]> * lint fix Signed-off-by: nayib-jose-gloria <[email protected]> * id fix Signed-off-by: nayib-jose-gloria <[email protected]> * id fix Signed-off-by: nayib-jose-gloria <[email protected]> * more ids Signed-off-by: nayib-jose-gloria <[email protected]> * id fix Signed-off-by: nayib-jose-gloria <[email protected]> * version id fix Signed-off-by: nayib-jose-gloria <[email protected]> * lint fix Signed-off-by: nayib-jose-gloria <[email protected]> * autogenerate migration script for db field changes + update README links Signed-off-by: nayib-jose-gloria <[email protected]> * update makefile test command documentation Signed-off-by: nayib-jose-gloria <[email protected]> * pr feedback on docs Signed-off-by: nayib-jose-gloria <[email protected]> * update migration script + db/local/load-schema + migration docs Signed-off-by: nayib-jose-gloria <[email protected]> * lint fixes Signed-off-by: nayib-jose-gloria <[email protected]> * remove unused imports Signed-off-by: nayib-jose-gloria <[email protected]> * tabs Signed-off-by: nayib-jose-gloria <[email protected]> * tabs Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> * feat: unpublished collections can be looked up by canonical id (#3907) * feat: unpublished collections can be looked up by canonical collection_id * complete docstring * change * valid -> invalid * fix: Documentation Tooltip on Icon Hover Only (#3893) * fix: icon doc hover only * using question mark SVG for tooltip icon hover * tooltip icon hover colors * fixing linting error "Type 'Element' is not assignable to type" * linting unused var * linting * importing svg directly and changing brightness Co-authored-by: atarashansky <[email protected]> * fix(coverage): adding code coverage (#3894) - add code coverage for python unit tests - run coverage for every PR and upload report to Codecov - add make recipes to generate coverage report - add allure test report for unit tests - update coverage version * fix: curation API differentiates unpublished collections for revisions (#3912) * Remove feature branch from push_test (#3915) * chore(redesign): Clean up processing container files (#3911) * Clean up old processing container files - adding json logs to processing container - clean up old processing container files * Update backend/layers/processing/process.py * remove old upload_failure, upload_success, and submission lambda code * feat(fmg): add analytics (#3896) * branch init * branch init * feat(fmg): add analytics Co-authored-by: Seve Badajoz <[email protected]> * feat(fmg): default to ttest (#3919) * branch init * feat(fmg): default to ttest Co-authored-by: Seve Badajoz <[email protected]> * fix processing logs (#3922) * feat(fmg): remove feature flag (#3943) * branch init * feat(fmg): remove feature flag Co-authored-by: Seve Badajoz <[email protected]> * feat: newly unpublished collections return canonical collection_id in API (#3944) * feat: newly published collections return canonical collection_id in the API * refactor to function * lint * add test for canonical id Co-authored-by: Trent Smith <[email protected]> * feat: newly unpublished collections return canonical collection_id in API (#3944) * feat: newly published collections return canonical collection_id in the API * refactor to function * lint * add test for canonical id Co-authored-by: Trent Smith <[email protected]> (cherry picked from commit ec2f61f) * chore: release main->staging (#4019) * fix: FMG icon not hiding for < 25 cells (#3946) * feat(curation API): newly unpublished collections return canonical collection_id in API (#3948) * feat: data submitters want collections to document their consortia (#3875) feat: data submitters want collections to document their consortia feat: add default empty consortia to create collection fix: fix texts for consortia on portal-api fix: update default for consortia to empty list feat: Add optional Consortia dropdown in New and Edit Collection with the required enumeration of approved consortia (#3642) feat: update edit collection form consortia dropdown with latest SDS dropdown component and style adjustments. fix: Linting and minor style updates. feat: added tests for updating consortia feat: validate consortia feat: add consortia to curation/discover API refactor: mock valid consortia fix: also override valid_consortia in business layer tests for #348 refactor: add explicit exception type for invalid collection metadata for #348 fix: fix format errors for #348 fix: fix lint errors for #348 fix: make consortia required on portal API collection response for #348 * fix: disable portal on consortia dropdown to maintain focus within create collection dialog. * refactor: break apart large tests * feat: added new discovery api test Co-authored-by: Fran McDade <[email protected]> * chore: cxg links are permalinks for new unpublished collections (#3957) * chore: cxg links are permalinks for new unpublished collections * curation API * rename * lint * chore: use canonical collection_id in curation API responses (#3964) * Fix curation post dataset for unpublished collections (#3966) * fix: support null metadata and double lookup for write datasets (#3968) * fix: support null metadata and double lookup for write datasets * PR changes * split test into 2 * fix: improve dataset identifiers (#3967) * fix: improve dataset identifiers * Linter * Test declutter * new test+overhaul * refactor: Refactor consortia type (#3973) (#3974) Co-authored-by: Fran McDade <[email protected]> * fix: local dev database config / secret name (#3985) Recent commit 4e6c4e2 simplified the '../database_local' secret name to '../database'. However, local dev case exceptions still caused the database config to look for a '../database_local' secret. * fix: handle null metadata in curation API GET collection (take 2) (#3986) * fix: handle null metadata case (take 2) * Linter * Update scripts/setup_dev_data.sh * fix: Portal Get /collections (#3980) * Fix Portal Get /collections - We need to return the canonical collection_id for a new unpublished collections when list all the collections. This technically isn't user facing problem but it would mean we no longer have working functional test if we don't fix this way. Since we do not openly disclose the version ID we wont be able to determine if a collection was successfully listed by get collection since creating a collection only return the canonical collection ID. * test: update the revision functional test to use the right explorer_url (#3993) * test: update the revision functional test to use the correct explorer_url * lint Co-authored-by: Trent Smith <[email protected]> * fix: add double lookup in dataset submissions lambda (#3994) * feat: adding search and details to dataset filter (#3905) * chore: POST /datasets returns the canonical dataset_id (#3999) * fix(curation api): GET /collections/{collection_id}/datasets/{dataset_id} (#3995) * fix(curation api): GET /collections/{collection_id}/datasets/{dataset_id} - added tests for revision_of in GET /collections/{collection_id}/datasets/{dataset_id} - revision_of only refers to the published dataset when revising a dataset. Otherwise it is NULL. - fix typing in test code * return the canonical dataset id when POSTing a dataset. * fix: check whether revision's consortia list has changed when determining whether collection has updates (#4003) Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> * chore: GET collections/:collection_id should return the correct dataset_id (#4006) * fix: add retries to dataset pipeline for failure and success handlers. (#4004) * fix: add retries to dataset pipeline for failure and success handlers. * fix typo * feat: cronjob to remove old rdev stacks (#4008) * fix: verion of happy-cleanup * fix: Adapt RevisionStatusTag to use redesigned data model (#4009) * fix: Adapt RevisionStatusTag to use redesigned data model Signed-off-by: nayib-jose-gloria <[email protected]> * fix 'updated' logic Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> * fix: stack name parsing bug This version update fixes a JSON parsing bug in the Github Action * docs: add valid consortia to swagger apis (#3997) (#4010) * fix: sort consortia on collection create or edit (#3996) (#4015) * fix: sort consortia on collection create or edit (#3996) * fix: update collection handles DOI correctly (#4017) * fix: check for Revision Changes accounts for deleted datasets in new data model + refactor Collection checks (#4016) * fix: check for Revision Changes accounts for deleted datasets in new data model + refactor Collection checks Signed-off-by: nayib-jose-gloria <[email protected]> * pr feedback Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> Co-authored-by: ashin-czi <[email protected]> Co-authored-by: David Rogers <[email protected]> Co-authored-by: Fran McDade <[email protected]> Co-authored-by: Emanuele Bezzi <[email protected]> Co-authored-by: Fran McDade <[email protected]> Co-authored-by: Daniel Hegeman <[email protected]> Co-authored-by: Nayib Gloria <[email protected]> Co-authored-by: Jake Heath <[email protected]> Signed-off-by: nayib-jose-gloria <[email protected]> Signed-off-by: Trent Smith <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Timmy Huang <[email protected]> Co-authored-by: Alex Lokshin <[email protected]> Co-authored-by: Emanuele Bezzi <[email protected]> Co-authored-by: Andrew Tolopko <[email protected]> Co-authored-by: Andrew Tolopko <[email protected]> Co-authored-by: Daniel Hegeman <[email protected]> Co-authored-by: Trent Smith <[email protected]> Co-authored-by: Trent Smith <[email protected]> Co-authored-by: SethFeingold <[email protected]> Co-authored-by: Timmy Huang <[email protected]> Co-authored-by: atarashansky <[email protected]> Co-authored-by: atarashansky <[email protected]> Co-authored-by: ashin-czi <[email protected]> Co-authored-by: atarashansky <[email protected]> Co-authored-by: Severiano Badajoz <[email protected]> Co-authored-by: Seve Badajoz <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: David Rogers <[email protected]> Co-authored-by: Fran McDade <[email protected]> Co-authored-by: Fran McDade <[email protected]> Co-authored-by: Jake Heath <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reviewers
Functional: @nayib-jose-gloria
Readability:
Changes
QA steps (optional)
Notes for Reviewer