remove "gated datasets unlock" logic #189
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
also: add two tests to ensure the gated datasets can be accessed
I changed (https://huggingface.co/datasets/severo/dummy_gated/commit/99194748bed3625a941aaf785740df02ca5762c9) severo/dummy_gated to a simpler dataset, without a python script, to avoid having non-related errors. Also in the commit: load the HF_TOKEN from a secret in https://github.com/huggingface/datasets-preview-backend/settings/secrets/actions to be able to run the unit tests
|
@SBrandeis I added you as a reviewer because it's part of the migration to the new tokens. I use it only to "unlock" the gated datasets (and later on, authenticate so that the server knows the dataset is unlocked). We don't access private datasets for now, so not related to public/private for now. |
|
By the way: what is the relationship between the token you gave me for dataset-preview-backend, and the gated datasets? Is the gated dataset unlocked for the token? for the user or org related to the token? |
|
unit tests should be failing due to #188 |
severo
changed the title
it's a dependency of lm-dataformat, and last version still depends on a vulnerable ujson version
the new "app" tokens on moonlanding can read the gated datasets without having to accept the conditions first, as it occurs for users. BREAKING CHANGE: 🧨 HF_TOKEN must be an app token
SBrandeis
approved these changes
Apr 1, 2022
|
Tests are failing tho |
|
yes, due to #188. See https://huggingface.slack.com/archives/C034N0A7H09/p1648817245991569 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
the new tokens are not related to a user and have read access to the gated datasets (right, @SBrandeis?).
also: add two tests to ensure the gated datasets can be accessed