Add ability to use custom CNN models #190
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
CustomModel
which allows using custom feature extractor. (Check example notebook:use_custom_model.ipynb
for detailed usage.)Efficientnet
andVision transfomer
cnn models in addition to the default MobilenetV3 for feature extraction.Why
MobilenetV3
does not represent state-of-the-art anymore when it comes to image classification tasks as evidenced by their performance on imagenet dataset. EfficientNet and Vision transformers instead form SOTA. (MobilenetV3
still remains as the default cnn model)How
CustomModel
has been added which can be passed to a newcnn
constructor argumentmodel_config
.CustomModel
accepts any torch module that can generate features. (a corresponding transform must also be provided that takes in a PIL image and outputs a pytorch tensor. The model must produce abatch_size x features
pytorch tensor). It can be imported fromimagededup.utils
.EfficientNet
andViT
models have been added toimagededup.utils.models
Choice of models
EfficientNet
andViT
have been added with the following in mind:torchvision.models
subpackage.The CustomModel can also use models that are not hosted on torchvision. To see an example of using a model from huggingface model hub, check out the example notebook
use_custom_model.ipynb
.