Enhance DatasetItem annotations for semantic segmentation model training use case

[vinnamkim]

Summary

• Ticket no. 141410
• Datumaro has been maintaining every mask as a binary mask representation. This is because of flexibility on manipulating dataset level management (e.g., label remapping). However, this design can bring performance degradation in case of model training use case. For example, semantic segmentation model training uses 2d mask filled in an integer value for each pixel. However, during importing a dataset, Datumaro should convert it to a collection of binary masks. It makes impossible to use the integer mask directly, so that it requires a 2d complexity computation cost to merge the imported binary masks into an integer mask.
• To resolve this performance degradation, this PR introduces a new interface on dataset_item.annotations by introducing Annotation(list[Annotation]) class. This class extends the pure Python list to equip more utility functions. At the same time, the utility function we introduce this time is dataset_item.annotations.get_semantic_seg_mask(). This function bypasses binary mask conversion to construct an integer mask used for semantic segmentation model training.

How to test

• Added unit tests to cover this change (mainly in tests/unit/test_annotation.py).
• This is a performance test on the semantic segmentation model training use case. Throughput improvement on getting a semantic segmentation mask is ~3x (31.38 it/s -> 99.31 it/s).
  

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have added the description of my changes into CHANGELOG.​
• I have updated the documentation accordingly

License

• I submit my code changes under the same MIT License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT

[codecov]

Codecov Report

Attention: Patch coverage is 79.31034% with 6 lines in your changes are missing coverage. Please review.

Please upload report for BASE (releases/1.7.0@3501e97). Learn more about missing BASE report.

Files	Patch %	Lines
src/datumaro/components/annotation.py	76.92%	4 Missing and 2 partials ⚠️

Additional details and impacted files

@@                Coverage Diff                @@
##             releases/1.7.0    #1503   +/-   ##
=================================================
  Coverage                  ?   80.78%           
=================================================
  Files                     ?      276           
  Lines                     ?    31511           
  Branches                  ?     6356           
=================================================
  Hits                      ?    25456           
  Misses                    ?     4643           
  Partials                  ?     1412           

Flag	Coverage Δ
ubuntu-20.04_Python-3.10	80.77% <79.31%> (?)
windows-2022_Python-3.10	80.75% <79.31%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[wonjuleee]

Why only Widerface is affected by this change?

[vinnamkim]

At this line, annotations: List is created

datumaro/src/datumaro/plugins/data_formats/widerface.py

Line 111 in 072c8a8

annotations = []

Then, this annotations is used to construct DatasetItem here

datumaro/src/datumaro/plugins/data_formats/widerface.py

Lines 123 to 128 in 072c8a8

	items[item_id] = DatasetItem(
	id=item_id,
	subset=self._subset,
	media=Image.from_file(path=image_path),
	annotations=annotations,
	)

However, in the following lines, Bbox is pushed into annotations, not items[item_id].annotations,

datumaro/src/datumaro/plugins/data_formats/widerface.py

Lines 160 to 169 in 072c8a8

	annotations.append(
	Bbox(
	float(bbox_list[0]),
	float(bbox_list[1]),
	float(bbox_list[2]),
	float(bbox_list[3]),
	attributes=attributes,
	label=label,
	)
	)

Previously, it should work because id(annotations) == id(items[item_id].annotations) (same Python object). However, after this PR change,

annotations: List
items[item_id].annotations: Annotations
=>
id(annotations) != id(items[item_id].annotations)

, so that widerface.py#L160-L169 above will be not working.