Export/Copy/Publish_Template : Structure & Code Only

[newton1985]

Long story short

As it stands, it is very difficult (and in many cases impossible) to export or publish a template from a large study. However, often it is not necessary to copy every file in a study, just the structural elements. I.e., the nodes, the connections between them, the directory structure of jupyter notebooks, and code/config files (*.py, *.ipynb, *.txt etc.). Therefore I propose we extend the export/publish options to allow the user to filter which data types they would like to consider when cloning the study.

Expected behaviour

I expect to be able to export a study's contents, clone a study, or publish a study template, without necessarily copying large data files that are generated during the execution of the pipeline. Ideally, the user could filter (or otherwise somehow specify) which files/filetypes they would like to preserve with the copy.

Actual behaviour

Currently publishing/exporting a study tries to zip everything, including potentially (very) large intermediate files that are produced during pipeline execution.

Note: your environment was attached but will not be displayed

[AntoninoMarioC]

In addition, it would be great to allow the user to export the pipeline in some latex standard formats for graphs, so that automatic generation of reports can be done as well as documentation.

[KZzizzle]

related to #558 and #466

[sanderegg]

There is now a function when creating a Template that allows for only copying the study skeleton without all its data. This is not yet with all filtering but can be a workaround:

[ignapas]

Export and duplication features will be disabled until we come up with a proper implementation.

[esraneufeld]

but the copy/publish/export without data remains available?

[pcrespov]

but the copy/publish/export without data remains available?
@esraneufeld

• publish will still be available (w/ and w/o data) (see screenshot above)
• export will be disabled (via config) until we have a more reliable solution
• duplicate (what you refer as copy) will be disabled (via config) until we have a more reliable solution

[esraneufeld]

@pcrespov
with the current data loss situation, i would very much appreciate, if we could at least keep the copy without data alive. what speaks against that?

[pcrespov]

@pcrespov with the current data loss situation, i would very much appreciate, if we could at least keep the copy without data alive. what speaks against that?

@esraneufeld That the duplicate (copy) is actually is implemented with export (the one we have disabled)

Nonetheless, we have a plan in place since a long time (and even some parts implemented) with a new design so it is time to retake it. SEE ITISFoundation/osparc-simcore#2152

[elisabettai]

@newton1985, it this one still high priority, given that you can at least create a template (with/without the data).

Maybe you can write down a smaller case that reflects the functionality has been already put in place.

[newton1985]

@elisabettai In the past this feature hasn't worked for larger studies, so to investigate the current status I gave it a try on osparc.io with Bladder Control as a test case - just the study structure, no data. It failed with the following message:

I guess this shouldn't happen since I requested to create a template without data.

[newton1985]

but assuming that the feature is working, I think it would satisfactorily address the need, and we could close the issue