Deprecate `ModelFilter/DatasetFilter` #2028

druvdub · 2024-02-15T01:53:35Z

Related to #1676

Moved all attributes from ModelFilter class to list_models arguments and added a filter query builder to build a query. Updated list_models to take both filter as an Iterable and also update the filter list based on other arguments in the method

…method Moved all attributes from ModelFilter class to list_models arguments and added a filter query builder to build a query. Updated list_models to take both filter as an Iterable and also update the filter list based on other arguments in the method

src/huggingface_hub/hf_api.py

Wauplin · 2024-02-15T08:41:52Z

Hi @druvdub! Thanks for the PR ❤️ I've tried to answer all of your questions in #1676 (comment). Please let me know if some parts are still unclear :)

Regarding this PR, would it be possible to fix the styling? The PR change diff is very big at the moment due to some whitespaces/newlines added or removed. This makes the PR quite difficult to review. You can fix this with the following steps:

install the dev dependencies:

pip install -e ".[dev]"

run the make style command (which runs ruff linter and formatter + a few scripts)

make style

(optional) once done, you should be able to run run quality without any error (checks styling + type annotations):

make quality

I believe the current changes were due to a "autoformat" or "format on save" feature in your IDE. Make sure to always run make style before committing changes :) Thanks in advance!

druvdub · 2024-02-15T10:27:25Z

Hi @druvdub! Thanks for the PR ❤️ I've tried to answer all of your questions in #1676 (comment). Please let me know if some parts are still unclear :)

Regarding this PR, would it be possible to fix the styling? The PR change diff is very big at the moment due to some whitespaces/newlines added or removed. This makes the PR quite difficult to review. You can fix this with the following steps:

install the dev dependencies:
pip install -e ".[dev]"
run the make style command (which runs ruff linter and formatter + a few scripts)
make style
(optional) once done, you should be able to run run quality without any error (checks styling + type annotations):
make quality
I believe the current changes were due to a "autoformat" or "format on save" feature in your IDE. Make sure to always run make style before committing changes :) Thanks in advance!

Yup no worries, for some reason the makefile did not work, So I had to now manually run the ruff checks

src/huggingface_hub/hf_api.py

… avoid duplication

src/huggingface_hub/utils/endpoint_helpers.py

src/huggingface_hub/hf_api.py

druvdub · 2024-02-15T14:26:18Z

@Wauplin I think this should be all in terms of functionality changes.

Should I update READMEs and completely remove ModelFilter/DatasetFilter because I believe it might affect the hf_api docs

And finally in terms of test changes should I just update the existing tests to take arguments instead of ModelFilter/DatasetFilter or write some completely new ones

Wauplin · 2024-02-15T16:32:38Z

Thanks for all the modifications and taking care of the special cases @druvdub! I'll have a look at it on tomorrow if that's fine with you :)

druvdub · 2024-02-15T16:54:43Z

Thanks for all the modifications and taking care of the special cases @druvdub! I'll have a look at it on tomorrow if that's fine with you :)

Yea sounds good to me :)

Wauplin

Thanks for the PR @druvdub! Logic-wise it looks good to me! 🔥 I left a couple of comments regarding the code + API but overall very nice one :) Once addressed, we will have to review some tests, especially if ModelFilter / DatasetFilter are used (since deprecation warning make the CI fail, on purpose). I can help on this part if needed (we have a expect_deprecation decorator for tests that are meant to break).

src/huggingface_hub/hf_api.py

src/huggingface_hub/utils/endpoint_helpers.py

src/huggingface_hub/hf_api.py

HuggingFaceDocBuilderDev · 2024-02-16T10:37:54Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

druvdub · 2024-02-16T15:54:14Z

Thanks for the PR @druvdub! Logic-wise it looks good to me! 🔥 I left a couple of comments regarding the code + API but overall very nice one :) Once addressed, we will have to review some tests, especially if ModelFilter / DatasetFilter are used (since deprecation warning make the CI fail, on purpose). I can help on this part if needed (we have a expect_deprecation decorator for tests that are meant to break).

Hey @Wauplin I have addressed all the comments so far, and hopefully I can look at the tests now.

In the meantime, is this expected?

And similarly do we need to update the README.md files as well

UPDATE:
the tests do fail, I can use the decorator but what I am thinking about are the additional scenarios
where we pass the the filter keyword arguments and ModelFilter as well. Do we need them assuming ModelFilter will be removed in sometime

This PR remove deprecated use of `ModelFilter`. Arguments can now be passed to `list_models` directly. See huggingface/huggingface_hub#2028 for more details.

…lFilter`) This PR remove deprecated import of `ModelFilter`. Arguments can now be passed to `list_models` directly. See huggingface/huggingface_hub#2028 for more details.

This PR remove deprecated use of `ModelFilter`. Arguments can now be passed to `list_models` directly. See huggingface/huggingface_hub#2028 for more details.

Hello 👋 This PR updates some attributes and methods that are now deprecated in the `huggingface_hub` library. - `modelId` is getting deprecated huggingface/huggingface_hub#2408 - `ModelFilter` has been deprecated for some time (see huggingface/huggingface_hub#2028) This PR will make the codebase more future-proof while being compatible with existing versions of `huggingface_hub`. Let me know if you have any questions 🤗

* Update hf_formatter.py (remove deprecated use of `ModelFilter`) This PR remove deprecated use of `ModelFilter`. Arguments can now be passed to `list_models` directly. See huggingface/huggingface_hub#2028 for more details. * fix: hf_formatter.py continue if exception --------- Co-authored-by: Prasad Chalasani <[email protected]>

This PR remove deprecated use of `ModelFilter`. Arguments can now be passed to `list_models` directly. See huggingface/huggingface_hub#2028 for more details.

druvdub marked this pull request as draft February 15, 2024 01:53

druvdub commented Feb 15, 2024

View reviewed changes

src/huggingface_hub/hf_api.py Outdated Show resolved Hide resolved

Reformat code

8f6df0e

druvdub changed the title ~~Deprecate ModelFilter/DatasetFilter~~ Deprecate ModelFilter/DatasetFilter Feb 15, 2024

Reformat Code

5278995

druvdub commented Feb 15, 2024

View reviewed changes

src/huggingface_hub/hf_api.py Outdated Show resolved Hide resolved

druvdub added 4 commits February 15, 2024 10:56

Update checks in to be consistent

e39e181

Add build_dataset_filter method and modified list_dataset

d757d8b

Update list_models and list_dataset methods to use set for filters to…

edd7d5a

… avoid duplication

Update warning to use FutureWarning instead of DeprecatedWarning

54210c1

druvdub commented Feb 15, 2024

View reviewed changes

src/huggingface_hub/utils/endpoint_helpers.py Show resolved Hide resolved

druvdub commented Feb 15, 2024

View reviewed changes

src/huggingface_hub/hf_api.py Outdated Show resolved Hide resolved

druvdub added 3 commits February 15, 2024 14:42

Merge branch 'main' into deprecate-model-filter-and-dateset-filter

f3c239e

Update search helper classes docstrings to follow HF doc-builder syntax

2fbaf9f

Modify warnings to be consistent

1919121

Wauplin reviewed Feb 16, 2024

View reviewed changes

Wauplin marked this pull request as ready for review February 16, 2024 10:30

Merge branch 'main' into deprecate-model-filter-and-dateset-filter

9d06abc

druvdub added 4 commits February 16, 2024 11:28

Move comments to inline to fix mypy issues

18707ed

Move model filter builder to

4804b44

Move dataset filter builder to list_datasets

8d84dad

Address comments and remove stacklevel argument from warnings

56f9e1b

Wauplin mentioned this pull request Aug 13, 2024

Update get_model_card.py (remove deprecated use of ModelFilter) asahi417/lm-question-generation#24

Merged

Wauplin mentioned this pull request Aug 13, 2024

Update hf_hub_utils.py (remove deprecated use of ModelFilter) alexandrainst/alexandra_ai_eval#139

Merged

Wauplin mentioned this pull request Aug 13, 2024

Remove deprecated import of ModelFilter llmware-ai/llmware#958

Merged

Wauplin mentioned this pull request Aug 13, 2024

Remove deprecated HfFolder ModelFilter and arguments n00mkrad/stable-diffusion-cust#4

Open

Wauplin mentioned this pull request Aug 13, 2024

Update huggingface_spider.py (remove deprecated use of ModelFilter) IDEA-CCNL/Fengshenbang-LM#465

Open

Wauplin mentioned this pull request Aug 13, 2024

Update __init__.py (remove deprecated use of ModelFilter) DavidMChan/grazier#4

Merged

Wauplin mentioned this pull request Aug 13, 2024

Update move_models.py (remove deprecated use of ModelFilter) asahi417/lm-vocab-trimmer#4

Merged

Wauplin mentioned this pull request Aug 13, 2024

Update hf_formatter.py (remove deprecated use of ModelFilter) langroid/langroid#540

Merged

Wauplin mentioned this pull request Aug 13, 2024

Update download.py (remove deprecated use of ModelFilter) gitmylo/audio-webui#243

Merged

Wauplin mentioned this pull request Aug 13, 2024

Update ChatHaruhi.py (remove deprecated use of ModelFilter) LC1332/Chat-Haruhi-Suzumiya#80

Open

Wauplin added a commit to Wauplin/LLM4SE that referenced this pull request Aug 13, 2024

Remove deprecated use of ModelFilter

2235d95

This PR remove deprecated use of `ModelFilter`. Arguments can now be passed to `list_models` directly. See huggingface/huggingface_hub#2028 for more details.

Wauplin mentioned this pull request Aug 13, 2024

Remove deprecated use of ModelFilter gai4se/LLM4SE#17

Merged

Wauplin added a commit to Wauplin/KVQuant that referenced this pull request Aug 13, 2024

Remove deprecated use of ModelFilter

2adb841

This PR remove deprecated use of `ModelFilter`. Arguments can now be passed to `list_models` directly. See huggingface/huggingface_hub#2028 for more details.

Wauplin mentioned this pull request Aug 13, 2024

Remove deprecated use of ModelFilter SqueezeAILab/KVQuant#13

Open

Wauplin mentioned this pull request Aug 22, 2024

ImportError: cannot import name 'ModelFilter' from 'huggingface_hub' #2478

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deprecate `ModelFilter/DatasetFilter` #2028

Deprecate `ModelFilter/DatasetFilter` #2028

druvdub commented Feb 15, 2024

Wauplin commented Feb 15, 2024

druvdub commented Feb 15, 2024 •

edited

Loading

druvdub commented Feb 15, 2024 •

edited

Loading

Wauplin commented Feb 15, 2024

druvdub commented Feb 15, 2024

Wauplin left a comment •

edited

Loading

HuggingFaceDocBuilderDev commented Feb 16, 2024

druvdub commented Feb 16, 2024 •

edited

Loading

Deprecate ModelFilter/DatasetFilter #2028

Deprecate ModelFilter/DatasetFilter #2028

Conversation

druvdub commented Feb 15, 2024

Wauplin commented Feb 15, 2024

druvdub commented Feb 15, 2024 • edited Loading

druvdub commented Feb 15, 2024 • edited Loading

Wauplin commented Feb 15, 2024

druvdub commented Feb 15, 2024

Wauplin left a comment • edited Loading

Choose a reason for hiding this comment

HuggingFaceDocBuilderDev commented Feb 16, 2024

druvdub commented Feb 16, 2024 • edited Loading

Deprecate `ModelFilter/DatasetFilter` #2028

Deprecate `ModelFilter/DatasetFilter` #2028

druvdub commented Feb 15, 2024 •

edited

Loading

druvdub commented Feb 15, 2024 •

edited

Loading

Wauplin left a comment •

edited

Loading

druvdub commented Feb 16, 2024 •

edited

Loading