Skip to content

Latest commit

 

History

History
84 lines (74 loc) · 5.05 KB

L2_validation_phase.mdx

File metadata and controls

84 lines (74 loc) · 5.05 KB
sidebar_position title
1
L2 - Validation phase

import * as Responsibilities from './Responsibilities.js';

Start situation

At the start of the validation phase, the virtual lab has the required components and the workflow has been developed and tested. Additionally the scientific (golden) user has made the data sets ready for the experiments.

During the validation phase

The golden user will conduct experiments in the virtual lab in this phase. The following should also be done phase:

  • Data
    • Make data fair.
    • Follow the community standards of the relevant domains for reading, writing and exchanging data.
  • Metadata
    • Complete all metadata fields.
  • Scenarios
    • Make sure and describe how the virtual lab can be used on different scenarios, i.e. other datasets and with other parameters.
  • Versioning
    • Add a version number to the virtual lab so users can refer to this number when they are reporting reproducibility or bug issues.
    • Give each containerized cell and executed workflow a persistent identifier and version number. Feature is currently under development. For progress, see #280.
  • Documentation
  • Workflow
    • Check the duration of computation, memory usage, and power usage of the containers.
  • Codebase
    • Add unit tests to verify the behavior of used methods and libraries.
    • Define clear responsibilities of all notebook cells, methods and classes.

Validation phase milestones

  • The submission for publication of a paper in the ecosystem domain presenting the scenarios run in the virtual lab.
  • The lab becoming publicly available.
  • Potentially, besides a domain paper, the submission for publication of a technical paper focusing on the infrastructure part of the virtual research environment.

Validation phase responsibilities

The following roles should be assigned during the validation phase:

  • Golden users: <Responsibilities.L2GoldenUser/>
  • Virtual lab core developers: <Responsibilities.L2CoreDeveloper/>
  • Community supporter: <Responsibilities.L2CommunitySupporter/>
  • Virtual lab trainer: <Responsibilities.L2Trainer/>
  • Infrastructure supporter: <Responsibilities.L2InfrastructureSupporter/>
  • VRE researcher: <Responsibilities.L2VREResearcher/>
  • Virtual lab coordinator: <Responsibilities.L2Coordinator/>
  • VRE DevOps engineer: <Responsibilities.L2DevOps/>

Exit condition

The virtual lab transitions to workshop use, once validation of the lab by doing scientific experiments has been done and a scientific paper is ready for publication.

It is important to find out who would be potential silver users of the virtual lab. In case no match between the virtual lab and potential silver users can be found, the virtual lab should be archived.

To advance to workshop use, all of the following boxes should be checked, in addition to the criteria for the initial proposal, and co-development:

  • User community
    • [ ] Other potential users have been identified.
  • Data
    • [ ] All input data of the lab is FAIR.
  • Metadata
    • [ ] All the fields of the metadata standard are present.
  • Scenarios
    • [ ] The virtual lab can be used in multiple scenarios, i.e. both the parameters and datasets can be changed to suit experiments of different researchers.
  • Documentation
    • [ ] There is a user manual and at least one domain scientist who was not involved in the development of the virtual lab has reviewed the user manual. The coding experience of the reviewer of the user manual is similar to the coding experience of the intended user.
    • [ ] How to use the virtual lab on a different scenario is explained.
  • Codebase
    • [ ] Unit tests verify the behavior of used methods and libraries. There should be a testing guideline, which will be done in this issue #274.
    • [ ] The virtual lab reads, writes and exchanges data in a way that meets domain-relevant community standards. Recommendations for what standards to use are under investigation, see github issue #281.
    • [ ] The code within cells is easily human-readable and others can easily modify it. If methods have side effects, this is clear to the user.
    • [ ] The input and output of each cell is clear. It is both clear what the structure is (e.g. what data type is used) and what the data content is from a domain perspective.
  • Workflow
    • [ ] The duration of computation, memory usage, and power usage of the containers is acceptable. As there is currently no dashboard to monitor resource usage, contact the VLIC team for guidelines.
  • Deployment
    • [ ] The virtual lab is publicly available.
  • Infrastructure
    • [ ] The infrastructure requirements for the workshop are known and the necessary infrastructure has been provided:
      • [ ] The number of people taking part in a workshop.
      • [ ] The random access memory and permanent storage usage of the virtual lab are known.
      • [ ] The amount of processors the virtual lab uses is known.