Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

♻️ Enhancement/storage refactoring part 1 #3077

Merged

Conversation

sanderegg
Copy link
Member

What do these changes do?

Split from and prequel to #3021, this changes are:

  • use dict/list/tuple instead of Dict/List/Tuple in all of storage
  • small refactoring
  • renamed fileId query parameter to file_id for consistency
  • removed extra_location/extra_source query parameters to get upload link (unused, @odeimaiz you might want to clean that out of the client code)

Related issue/s

How to test

Checklist

@sanderegg sanderegg added this to the Croissant milestone May 30, 2022
@sanderegg sanderegg requested review from pcrespov and GitHK as code owners May 30, 2022 11:15
@sanderegg sanderegg self-assigned this May 30, 2022
@sanderegg sanderegg requested a review from mguidon as a code owner May 30, 2022 11:15
@sanderegg sanderegg force-pushed the enhancement/storage_refactoring_step1 branch from 61fc76e to fd6e8b7 Compare May 30, 2022 11:15
@codecov
Copy link

codecov bot commented May 30, 2022

Codecov Report

Merging #3077 (e8d7861) into master (4ef961e) will increase coverage by 0.0%.
The diff coverage is 85.1%.

Impacted file tree graph

@@          Coverage Diff           @@
##           master   #3077   +/-   ##
======================================
  Coverage    80.9%   80.9%           
======================================
  Files         716     716           
  Lines       30896   30888    -8     
  Branches     4031    4030    -1     
======================================
- Hits        25017   25012    -5     
- Misses       5004    5006    +2     
+ Partials      875     870    -5     
Flag Coverage Δ
integrationtests 66.2% <100.0%> (+<0.1%) ⬆️
unittests 76.8% <85.1%> (+<0.1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...-library/src/models_library/api_schemas_storage.py 100.0% <ø> (ø)
...s/storage/src/simcore_service_storage/db_tokens.py 89.6% <ø> (-0.4%) ⬇️
...vice_webserver/exporter/formatters/formatter_v1.py 78.8% <ø> (ø)
.../src/simcore_service_webserver/storage_handlers.py 84.9% <ø> (ø)
...service_storage/datcore_adapter/datcore_adapter.py 32.0% <16.6%> (ø)
...es/storage/src/simcore_service_storage/handlers.py 73.6% <87.5%> (+1.0%) ⬆️
...torage/src/simcore_service_storage/access_layer.py 72.7% <100.0%> (ø)
services/storage/src/simcore_service_storage/db.py 71.4% <100.0%> (ø)
...ervices/storage/src/simcore_service_storage/dsm.py 72.2% <100.0%> (ø)
...ices/storage/src/simcore_service_storage/models.py 78.8% <100.0%> (ø)
... and 7 more

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


if query.get("extra_source") and query.get("extra_location"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I guess this is no longer used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was not used for years

Comment on lines 66 to 68
start = perf_counter()
print(f"--> uploading {file=}")
async with ClientSession() as session:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: I see no timeouts here. What if remote has an issue and hangs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, but this is a pytest helper. anyway will remove it from this PR since it uses stuff that is not available in this PR...

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@sanderegg sanderegg merged commit 9ea5d8d into ITISFoundation:master May 30, 2022
@sanderegg sanderegg deleted the enhancement/storage_refactoring_step1 branch May 30, 2022 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants