Add handling of multiclass format in detection dataset loading #4

aminemindee · 2022-09-01T13:05:28Z

Hello,

For testing,

dataset, in tests/common/test_datasets.py
preditions , in tensorflow and pytorch in test_models_detection and test_models_zoo
for training part, you should use the command

python references/detection/train_pytorch.py path/to/train/folder/ /path/to/test/folder/ "db_resnet34" for db model
python references/detection/train_pytorch.py path/to/train/folder/ /path/to/test/folder/ "linknet_resnet34" for linknet model

tests/common/test_models_builder.py

odulcy-mindee · 2022-09-29T09:37:58Z

doctr/datasets/datasets/base.py

+                for k, v in target.items():
+                    img, target[k] = self.sample_transforms(img, v)


Here, you apply self.sample_transforms multiple time on img. Also, img seems to be strongly linked to the latest target in dict because it's the output of self.sample_transforms with v as target, but I'm not sure what is the impact on img.

it's more the other way around from what i see the transforms used in training are resizing transform and it's the target changes that is linked to the image

Yes the changes are done multiple time to the img

Sorry, I didn't understand. Do you have an example ?
Is it fine to apply multiple times the same transformation on the same image ?

Yes you're right ! didn't see it

i tried resolving it this way

img_transformed = img.copy() for class_name, bboxes in target.items(): img_transformed, target[class_name] = self.sample_transforms(img, bboxes) img = img_transformed

what do you think ?

doctr/datasets/datasets/pytorch.py

doctr/datasets/datasets/tensorflow.py

doctr/datasets/detection.py

doctr/models/_utils.py

doctr/models/detection/differentiable_binarization/base.py

doctr/models/detection/differentiable_binarization/pytorch.py

odulcy-mindee · 2022-09-29T12:48:58Z

references/detection/train_tensorflow.py

-            val_metric.update(gts=boxes_gt, preds=boxes_pred[:, :4])
+        for target, loc_pred in zip(targets, loc_preds):
+            if isinstance(target, np.ndarray):
+                target = {"words": target}


Constant words

references/detection/train_tensorflow.py

tests/common/test_models_builder.py

tests/pytorch/test_models_detection_pt.py

tests/tensorflow/test_models_detection_tf.py

doctr/datasets/detection.py

doctr/datasets/utils.py

doctr/models/_utils.py

doctr/models/detection/linknet/base.py

doctr/utils/visualization.py

references/detection/train_pytorch.py

tests/common/test_models_builder.py

odulcy-mindee · 2022-10-10T09:44:58Z

doctr/utils/visualization.py

@@ -141,6 +141,24 @@ def create_obj_patch(
    raise ValueError("invalid geometry format")


+def get_colors(num_colors: int) -> List:


Typing on return statement: List[Tuple[int, int, int]]

odulcy-mindee · 2022-10-10T09:51:14Z

doctr/datasets/datasets/base.py

@@ -56,8 +56,10 @@ def __getitem__(self, index: int) -> Tuple[Any, Any]:

        if self.sample_transforms is not None:
            if isinstance(target, dict):
+                img_transformed = img.copy()


.copy() didn't work neither on Tensorflow nor PyTorch. You may need to check if the current backend (TF or PyTorch) and use the good method to clone the tensor

odulcy-mindee · 2022-10-10T09:53:31Z

tests/common/test_models_builder.py

+    boxes = {CLASS_NAME: np.random.rand(words_per_page, 6)}
+    boxes[CLASS_NAME][:2] *= boxes[CLASS_NAME][2:4]


You should remove these statements, otherwise boxes parameter is not used ;-)

odulcy-mindee

Awesome work ! 🚀

…with all transforms

…handle better class ids

…removed from CI

…impler

…de from it

…as dict

…ith it

…ates

…lements for kie predictor (#6) * feat: ✨ add load backbone * feat: change kie predictor out * fix new elements for kie, dataset when class is empty and fix and add tests * fix api kie route * fix evaluate kie script * fix black * remove commented code * update README

aminemindee force-pushed the feat/multiclass branch from ad3181b to f9996cd Compare September 8, 2022 13:13

aminemindee force-pushed the feat/multiclass branch 2 times, most recently from e33889b to 2e2a281 Compare September 22, 2022 15:02

aminemindee requested a review from odulcy-mindee September 26, 2022 09:51

odulcy-mindee reviewed Sep 26, 2022

View reviewed changes

tests/common/test_models_builder.py Outdated Show resolved Hide resolved

aminemindee force-pushed the feat/multiclass branch from 2e2a281 to 3c334cd Compare September 26, 2022 10:03

odulcy-mindee reviewed Sep 29, 2022

View reviewed changes

odulcy-mindee reviewed Oct 3, 2022

View reviewed changes

odulcy-mindee reviewed Oct 10, 2022

View reviewed changes

odulcy-mindee approved these changes Oct 11, 2022

View reviewed changes

aminemindee force-pushed the feat/multiclass branch from eccb2ee to 8965394 Compare October 12, 2022 13:38

aminemindee mentioned this pull request Oct 13, 2022

Feat: Make detection training and inference Multiclass mindee/doctr#1097

Merged

5 tasks

aminemindee added 17 commits December 5, 2022 17:05

feat: add handling of multiclass format in detection dataset loading …

24d7009

…with all transforms

feat: commit to rebase

b713128

fix: fix loss computation and make training work

64dc994

feat: make loss computation vectorized and change target building to …

95c017e

…handle better class ids

add multi class intergration in prediction pipeline

f630280

feat: add multiclass to pytorch and fix tests

1563f1e

fix: fix all pr reviews

80475ac

fix: second review comments

fa887c2

fix: pr comments

31165b5

fix: fix CI evaluate script

6cef57c

feat: add doc about Pages changes and multilabel dataset for training

79a320a

feat: fix api dockerfile and make it work with new changes

863fd98

fix reference tests

fcee26a

fix: fix api doc and dockerfile to create requirement txt inside and …

cb19734

…removed from CI

fix api docker with doctr 0.6.0

133316f

Up major version to 1.0.0

75c1cd3

refactor: refactor invert dict list and list dict function into one s…

1cc2204

…impler

aminemindee and others added 23 commits December 5, 2022 17:10

fix: style and mypy

3b22ebe

docs: make it more clear for new data format

3747094

explain why python version was upped

a96b83c

add assert on length of tuple

b48a243

add doc and simple name to invert data structure function

f5f4dce

feat: add class names can be obtained from model config

47bcdbf

fix: prioritize class_names from dataset over model config

b8c1418

fix: fix show samples in training

b9810a0

fix: add check when target is dict and all values are numpy arrays

829e266

fix: make detection target always dict and remove unnecessary made co…

223722d

…de from it

fix: script detection evaluation tests and dataset tests with target …

f4f4aed

…as dict

fix tests also on pytorch

be40ef6

feat: Add kie predictor and io elements and visualization that come w…

2d24441

…ith it

fix: revert ocr predictor to old format

20bfd41

fix tests and add test for kie predictor

b07abc0

up project version to 0.7.0

b74d7f2

update api to fix it and add kie route

f3c5d65

fix api version

aac4347

feat: sort class names to always have the same order.

d112c2d

sort imports to avoid cyclic imports

d8991d9

fix class_names default, use of tf_is_available avoid and copyright d…

2442a4f

…ates

feat: update readme and doc with kie predictor

7cb23dd

aminemindee force-pushed the feat/multiclass branch from aa15f43 to b396f6d Compare December 5, 2022 16:11

fix mypy

73d10a0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add handling of multiclass format in detection dataset loading #4

Add handling of multiclass format in detection dataset loading #4

aminemindee commented Sep 1, 2022 •

edited

Loading

odulcy-mindee Sep 29, 2022

aminemindee Oct 3, 2022

odulcy-mindee Oct 3, 2022

aminemindee Oct 5, 2022

odulcy-mindee Sep 29, 2022

aminemindee Oct 3, 2022

odulcy-mindee Oct 10, 2022

aminemindee Oct 11, 2022

odulcy-mindee Oct 10, 2022

odulcy-mindee Oct 10, 2022

aminemindee Oct 11, 2022

odulcy-mindee left a comment

		for k, v in target.items():
		img, target[k] = self.sample_transforms(img, v)

		@@ -141,6 +141,24 @@ def create_obj_patch(
		raise ValueError("invalid geometry format")


		def get_colors(num_colors: int) -> List:

		boxes = {CLASS_NAME: np.random.rand(words_per_page, 6)}
		boxes[CLASS_NAME][:2] *= boxes[CLASS_NAME][2:4]

Add handling of multiclass format in detection dataset loading #4

Are you sure you want to change the base?

Add handling of multiclass format in detection dataset loading #4

Conversation

aminemindee commented Sep 1, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

odulcy-mindee left a comment

Choose a reason for hiding this comment

aminemindee commented Sep 1, 2022 •

edited

Loading