Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to use custom CNN models #190

Merged
merged 15 commits into from
Apr 28, 2023
Merged

Add ability to use custom CNN models #190

merged 15 commits into from
Apr 28, 2023

Conversation

tanujjain
Copy link
Collaborator

What

  • Add a construct CustomModel which allows using custom feature extractor. (Check example notebook: use_custom_model.ipynb for detailed usage.)
  • Add Efficientnet and Vision transfomer cnn models in addition to the default MobilenetV3 for feature extraction.

Why

  • MobilenetV3 does not represent state-of-the-art anymore when it comes to image classification tasks as evidenced by their performance on imagenet dataset. EfficientNet and Vision transformers instead form SOTA. ( MobilenetV3 still remains as the default cnn model)
  • The CNN ecosystem has many promising models out- several hosted on torchhub, huggingface modelhub, etc. It would be great to allow the community to leverage a cnn model of their choice for their usecase.

How

  • A construct CustomModel has been added which can be passed to a new cnn constructor argumentmodel_config. CustomModel accepts any torch module that can generate features. (a corresponding transform must also be provided that takes in a PIL image and outputs a pytorch tensor. The model must produce a batch_size x features pytorch tensor). It can be imported from imagededup.utils.
  • EfficientNet and ViT models have been added to imagededup.utils.models

Choice of models

EfficientNet and ViT have been added with the following in mind:

  1. Availability on torchvision.models subpackage.
  2. Size of the model- wanted to avoid packaging models that are too large (too many params = large memory requirements and increased feature generation times.)
  3. SOTA on imagenet dataset.

The CustomModel can also use models that are not hosted on torchvision. To see an example of using a model from huggingface model hub, check out the example notebook use_custom_model.ipynb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant