[RFC Discussion] Control of data outputs

[M-casado]

Start Date

2024-12-05

Reference Issues

GenomicDataInfrastructure/standard-operating-procedures#19

Summary

This RFC proposes the creation of a node-specific SOP titled "Control of Data Outputs". The SOP will provide a standardised protocol for managing user requests for data extraction after analysis. This includes validation, review, and secure release of data while ensuring compliance with data access requirements and privacy standards.

Use-case and Basic Example

When a user completes a federated analysis on a GDI node and requests to extract the resulting data, the SOP will guide the node in processing this request securely and consistently.

Example:

• A researcher requests a dataset export post-analysis.
• The node locks the dataset to prevent modifications, verifies the researcher’s permissions, and reviews the data for compliance (e.g., anonymisation).
• Upon approval, the dataset is securely exported, and the request is logged for auditing purposes.

Motivation

This SOP is necessary to:

1. Ensure consistency: Different nodes may currently handle data outputs differently, leading to inconsistencies in compliance and user experience.
2. Support compliance: A standardised protocol ensures adherence to data access agreements, privacy laws (e.g., GDPR), and security standards.
3. Enhance trust: Establishing a clear, auditable process enhances trust among users and stakeholders.
   Without this SOP, nodes risk unintentional data breaches, non-compliance with agreements, and inefficient data output handling.

Detailed Design

The SOP could include the following steps:

This SOP may be tightly related to GDI-SOP0003_SOP-1+MG-DAC-Recommendation-Approval.md.

1. Request Validation:

   • Verify user credentials and data access permissions.
   • Cross-check the request with the relevant Data Access Agreement (DAA).

2. Data Locking:

   • Lock the requested dataset to prevent modifications during the review process.
   • Mark the dataset. Maybe with "Under Export Review", for tracking.

3. Data Review:

   • Ensure compliance with privacy requirements (e.g., anonymisation, aggregation).
   • Validate that the requested data matches the user's authorised access scope. Done automatically based on AAI, I guess.

4. Approval Process:

   • Obtain necessary approvals from designated governance bodies (e.g., GDI's EDIC?).

5. Secure Data Release:

   • Export the dataset in a secure format (e.g., encrypted files).
   • Notify the user and provide access details.
   • Ensure the activity is logged, for future audits.

6. Audit and Monitoring:

   • Log all actions related to the request, review, and release.
   • Conduct periodic audits to ensure SOP compliance (?).

Acceptance Criteria

• All GDI nodes must implement the SOP or a compatible protocol.
• SOP must support logging and auditing of all actions.
• Users must receive data outputs securely and in compliance with agreements.
• Nodes must ensure that data integrity is maintained throughout the process.

Drawbacks

• Implementation complexity: Nodes with limited resources may face challenges in adopting the SOP, particularly the manual steps.
• Operational overhead: The locking and review processes may introduce delays in data retrieval for users.
• Potential resistance: Nodes may hesitate to adopt the SOP if their current systems are significantly different or if transition costs are high.

Alternatives

1. Node-Specific Solutions:

   • Nodes could make their own instances of this SOP, as a Node-specific GDI SOP.

2. Centralised Data Review:

   • A central GDI committee could handle all data requests (similar to how the EDIC is suppose to handle data access requests?).

3. No SOP:

   • Not implementing this SOP could result in continued variability in how data outputs are managed.

Adoption Strategy

No response

Unresolved Questions

• What kind of analysis output is in scope for this SOP? Is it a dataset that the user was granted access to, then applied analysis in a TRE, and then wants to export their results?
• How should node-specific requirements be handled while maintaining standardisation? How to converge different processing environments?
• Should there be a threshold for request complexity that requires escalation to GDI-level approval? Should all be automated?
• What tools or platforms should be recommended to ensure secure data exports? Similar to the GDI Starter kit data retrieval?
• Should the SOP allow for exceptions in emergency cases, and how would those be managed?

[M-casado]

@brunopacheco1 - Can you lend me a hand in choosing who to bring to this discussion about this SOP? Correct me if I'm wrong, but perhaps this would be part of the user-portal?

Again, there are more questions than answers regarding this SOP, but that's what we ought to discuss about.

[brunopacheco1]

What could be eventually in the user portal is the aftermath of the analysis. Both data analysis and virtual cohort/conbined datasets, so the data user can cite them in their research.

[M-casado]

Perhaps you have a SOP regarding data onboarding

We do have something similar in development: GenomicDataInfrastructure/standard-operating-procedures#10

Also, if it's not through the User Portal, what method would GDI users have to extract the outputs of the analyses? Is there a team you may have in mind I could poke to lend a hand here?

[brunopacheco1]

@M-casado sorry for the delay. I will try to setup a workshop to findout the role of the User Portal in that process. I don't know, we may need people from Pillar I, and Pillar III(?). Would you like to be involved?

[M-casado]

Sure. My role in all this is to craft the base of the SOP and let people that know much more than me input their thoughts in it. I saw your email now. Before we organise something big, it's important to have the discussion on:

• How each of us see this SOP
• What are the requirements for it
• Who may be able to contribute with their expertise

I think, once we have that, we can target a workshop for it.

[M-casado]

Hi @brunopacheco1, just checking in after the Christmas break: did we have something planned? Anyone else that would be fit for this SOP development if not the User Portal?

[M-casado]

The initial description of the SOP we obtained was:

SOP Title	Control of Data Outputs
Purpose	Some users may want to extract data once analysis has been completed. The purpose of this SOP is to define and standardise the protocol for this.
Scope	This SOP details the protocol to be followed when a user requests output of data.
Procedure	- Lock the data to be exported against writing access
	- Control data based on data access requirements
	- Release data

[M-casado]

And the SOP request in the GDI SOP repository is GenomicDataInfrastructure/standard-operating-procedures#19