Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Requirements for data loaders for scivision #511

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Conversation

ots22
Copy link
Member

@ots22 ots22 commented Jan 26, 2023

No description provided.

@ots22 ots22 marked this pull request as draft January 26, 2023 14:56
Copy link
Member

@acocac acocac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add data loader feature suggested by @pyatsysh in today scivision coworking session

docs/scip/0004-data-loading.md Show resolved Hide resolved
docs/scip/0004-data-loading.md Show resolved Hide resolved

- Often locations are specified using URLs with a http/https scheme, but e.g. directory browsing is not supported by http, which limits the generality or usefulness of this approach.

- One possibility that is supported by plain http is a direct link to 'archive' file system (e.g. a zip file containing one of the patterns below).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This way does seem like it would be easy for a data provider to comply with

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is at least partly supported already, but it seemed worth being explicit about the issue, and that a zip file (or other archive) is available as a single-file option. One motivation for having special handling of other data hosting services is so data providers have a better chance of including their data as-is, but with this option as a fallback, potentially.

docs/scip/0004-data-loading.md Show resolved Hide resolved
@evangeline-corcoran evangeline-corcoran added this to the Core features milestone Apr 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

4 participants