Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicate should use the mechanism of the templates instead of exporting/importing #2152

Closed
sanderegg opened this issue Feb 17, 2021 · 6 comments
Assignees
Labels
a:storage issue related to storage service a:webserver issue related to the webserver service t:enhancement Improvement or request on an existing feature

Comments

@sanderegg
Copy link
Member

sanderegg commented Feb 17, 2021

Current duplication of projects in the backend works as:

  • export the project to duplicate in the webserver service in /tmp
  • re-import that project

This is ok for small project size but for bigger one is going to be:

  • an issue maybe with storage availability in the webserver
  • unnecessary creation of data that could be very big depending on the projects

For duplication I think the best way is to go through the same path as when a project is created from a template. but doing it from a standard project instead. Most of the mechanisms should already be available.
Pros are:

  1. directly duplicates the data in the DB, S3 without first downloading and uploading
    Some things that might need to be taken in account are:
  2. duplication of the nodes runHash in case of nodes that ran already successfully must be handled (like for the current importer)
  3. handling of the files eTag (might actually also be a problem with duplication from template)

Possible improvements

  • maybe think of copy on modification instead of copying everything
@sanderegg sanderegg added a:webserver issue related to the webserver service t:enhancement Improvement or request on an existing feature labels Feb 17, 2021
@pcrespov pcrespov added the a:storage issue related to storage service label Feb 17, 2021
@pcrespov
Copy link
Member

I fully agree. All operations involving copying studies should follow similar path, i.e.

  • project document duplicated in database: currently, responsibility of projects module app, which will evolve into a separate service at some point ....
  • study data duplicated in s3 : delegated to storage (referred in point 1)

both operations can be coordinated by the projects module app but SHALL NOT be executed by the webserver.

@pcrespov
Copy link
Member

@GitHK please comment on this

@mguidon
Copy link
Member

mguidon commented Feb 18, 2021

This sounds very reasonable. I concur with @sanderegg suggestion.

@pcrespov
Copy link
Member

pcrespov commented Feb 9, 2022

consider to implement ITISFoundation/osparc-issues#560

@pcrespov pcrespov self-assigned this Feb 9, 2022
@pcrespov pcrespov added this to the R.Schumann milestone Feb 9, 2022
@pcrespov
Copy link
Member

Will resolve bug #558

@sanderegg
Copy link
Member Author

this is now implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:storage issue related to storage service a:webserver issue related to the webserver service t:enhancement Improvement or request on an existing feature
Projects
None yet
Development

No branches or pull requests

3 participants