[bug] Open json file & json folder, file list panel is empty #1009

HowcanoeWang · 2022-04-07T04:06:50Z

Labelem version: 5.0.0

When open dir with the following structure (with only json files):

The file list panel keeps empty

When open specific json name, the image appears but the file list panel still empty:

Expected:

The file panel appear with json image name, the left status box checked (means has been labelled)

wkentaro · 2022-06-25T04:25:21Z

What happens when you do the following?

git clone https://github.com/wkentaro/labelme
labelme labelme/examples/semantic_segmentation/data_annotated/

HowcanoeWang · 2022-06-27T07:07:49Z

What happens when you do the following?

git clone https://github.com/wkentaro/labelme
labelme labelme/examples/semantic_segmentation/data_annotated/

Cloning to local, pip install -r requirements-dev.txt, and then python setup.py build -> python setup.py install, and run labelme labelme/examples/semantic_segmentation/data_annotated/

Get the following error:

Traceback (most recent call last):
  File "C:\ProgramData\Miniconda3\envs\labelme\Scripts\labelme-script.py", line 33, in <module>
    sys.exit(load_entry_point('labelme==5.0.1', 'console_scripts', 'labelme')())
  File "C:\ProgramData\Miniconda3\envs\labelme\Scripts\labelme-script.py", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "C:\ProgramData\Miniconda3\envs\labelme\lib\importlib\metadata.py", line 86, in load
    module = import_module(match.group('module'))
  File "C:\ProgramData\Miniconda3\envs\labelme\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "C:\ProgramData\Miniconda3\envs\labelme\lib\site-packages\labelme-5.0.1-py3.9.egg\labelme\__init__.py", line 17, in <module>
    QT4 = QT_VERSION[0] == "4"
TypeError: 'NoneType' object is not subscriptable

wkentaro · 2022-06-27T09:05:57Z

I'm not sure why this happens, but it looks like pyqt is not installed properly. Maybe you need to run pip install pyqt5?

HowcanoeWang · 2022-06-27T09:27:15Z

(labelme) E:\Github\labelme>pip install pyqt5
Requirement already satisfied: pyqt5 in c:\programdata\miniconda3\envs\labelme\lib\site-packages\pyqt5-5.15.7-py3.9-win-amd64.egg (5.15.7)
Requirement already satisfied: PyQt5-sip<13,>=12.11 in c:\programdata\miniconda3\envs\labelme\lib\site-packages\pyqt5_sip-12.11.0-py3.9-win-amd64.egg (from pyqt5) (12.11.0)
Requirement already satisfied: PyQt5-Qt5>=5.15.0 in c:\programdata\miniconda3\envs\labelme\lib\site-packages\pyqt5_qt5-5.15.2-py3.9-win-amd64.egg (from pyqt5) (5.15.2)

Here is my environment:

(labelme) E:\Github\labelme>pip list
Package            Version
------------------ ---------
atomicwrites       1.4.0
attrs              21.4.0
black              22.1.0
bleach             5.0.0
certifi            2022.6.15
charset-normalizer 2.0.12
click              8.1.3
colorama           0.4.5
commonmark         0.9.1
cycler             0.11.0
docutils           0.18.1
flake8             3.8.4
github2pypi        1.0.0
hacking            4.1.0
idna               3.3
imgviz             1.5.0
importlib-metadata 4.12.0
iniconfig          1.1.1
keyring            23.6.0
kiwisolver         1.4.3
labelme            5.0.1
matplotlib         3.2.2
mccabe             0.6.1
mypy-extensions    0.4.3
natsort            8.1.0
numpy              1.23.0
packaging          21.3
pathspec           0.9.0
Pillow             9.1.1
pip                21.2.4
pkginfo            1.8.3
platformdirs       2.5.2
pluggy             1.0.0
py                 1.11.0
pycodestyle        2.6.0
pyflakes           2.2.0
Pygments           2.12.0
pyparsing          3.0.9
PyQt5              5.15.7
PyQt5-Qt5          5.15.2
PyQt5-sip          12.11.0
pytest             7.1.2
pytest-qt          4.1.0
python-dateutil    2.8.2
pywin32-ctypes     0.2.0
PyYAML             6.0
QtPy               2.1.0
readme-renderer    35.0
requests           2.28.0
requests-toolbelt  0.9.1
rfc3986            2.0.0
rich               12.4.4
setuptools         61.2.0
six                1.16.0
termcolor          1.1.0
tomli              2.0.1
twine              4.0.1
typing_extensions  4.2.0
urllib3            1.26.9
webencodings       0.5.1
wheel              0.37.1
wincertstore       0.2
zipp               3.8.0

I think similar with this issue:
#1011 (comment)

wkentaro · 2022-06-27T10:10:55Z

Have you tried this?

#1011 (comment)

What does conda list says?

HowcanoeWang · 2022-06-27T11:18:40Z

Please skip this and check the next comment

create new conda env [failed]

E:\Github\labelme> conda create -n lbtest python=3 qt pyqt pyside2 -y
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - pyside2

install qt by conda [failed]

And also tried to install qt via conda (I prefer using conda as virtual env manager and using pip install only, rather than using any conda install):

(labelme) E:\Github\labelme>conda install qt
Collecting package metadata (current_repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.10.3
  latest version: 4.13.0

Please update conda by running

    $ conda update -n base -c defaults conda



## Package Plan ##

  environment location: C:\ProgramData\Miniconda3\envs\labelme

  added / updated specs:
    - qt


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    icu-58.2                   |       ha925a31_3         9.4 MB
    jpeg-9e                    |       h2bbff1b_0         292 KB
    qt-5.9.7                   |   vc14h73c81de_0        72.5 MB
    zlib-1.2.12                |       h8cc25b3_2         116 KB
    ------------------------------------------------------------
                                           Total:        82.3 MB

The following NEW packages will be INSTALLED:

  icu                pkgs/main/win-64::icu-58.2-ha925a31_3
  jpeg               pkgs/main/win-64::jpeg-9e-h2bbff1b_0
  libpng             pkgs/main/win-64::libpng-1.6.37-h2a8f88b_0
  qt                 pkgs/main/win-64::qt-5.9.7-vc14h73c81de_0
  zlib               pkgs/main/win-64::zlib-1.2.12-h8cc25b3_2


Proceed ([y]/n)? y


Downloading and Extracting Packages
qt-5.9.7             | 72.5 MB   | ############################# | 100%
icu-58.2             | 9.4 MB    | ############################# | 100%
zlib-1.2.12          | 116 KB    | ############################# | 100%
jpeg-9e              | 292 KB    | ############################# | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

(labelme) E:\Github\labelme>labelme
Traceback (most recent call last):
  File "C:\ProgramData\Miniconda3\envs\labelme\Scripts\labelme-script.py", line 33, in <module>
    sys.exit(load_entry_point('labelme==5.0.1', 'console_scripts', 'labelme')())
  File "C:\ProgramData\Miniconda3\envs\labelme\Scripts\labelme-script.py", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "C:\ProgramData\Miniconda3\envs\labelme\lib\importlib\metadata.py", line 86, in load
    module = import_module(match.group('module'))
  File "C:\ProgramData\Miniconda3\envs\labelme\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "C:\ProgramData\Miniconda3\envs\labelme\lib\site-packages\labelme-5.0.1-py3.9.egg\labelme\__init__.py", line 17, in <module>
    QT4 = QT_VERSION[0] == "4"
TypeError: 'NoneType' object is not subscriptable

Please skip this, and check the next comment

seems pyqt5 problem [solution not found]

(labelme) E:\Github\labelme>python
Python 3.9.12 (main, Apr  4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import PyQt5
>>> from PyQt5.QtCore import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: DLL load failed while importing QtCore: The specified procedure could not be found.

Following this website, copy python3.dll and python39.dll to virtual env scripts folder seems still not work

Follwoing this website, using admin cmd to run >python -m pip install --upgrade pyqt5 and conda update --all, still not work.
PS: the pyqt5==5.9 in previous link has been removed from pip list:

(labelme) C:\Users\CREST>python -m pip install --upgrade pyqt5==5.9
ERROR: Could not find a version that satisfies the requirement pyqt5==5.9 (from versions: 5.12.3, 5.14.0, 5.14.1, 5.14.2, 5.15.0, 5.15.1, 5.15.2, 5.15.3, 5.15.4, 5.15.5, 5.15.6, 5.15.7)
ERROR: No matching distribution found for pyqt5==5.9

HowcanoeWang · 2022-06-27T11:38:31Z

Succeed by the following:

(labelme) C:\Users\CREST>conda create -c conda-forge -n lbtest python=3 qt pyqt pyside2
(labelme) C:\Users\CREST>conda activate lbtest
# cd to path
(lbtest) E:\Github\labelme>python setup.py build
running build
running build_py

(lbtest) E:\Github\labelme>python setup.py install
...
UPDATING build\lib.win-amd64-cpython-310\matplotlib\_version.py
set build\lib.win-amd64-cpython-310\matplotlib\_version.py to '3.2.2'
checkdep_freetype2.c
src/checkdep_freetype2.c(5): fatal error C1189: #error:  "FreeType version 2.3 or higher is required. You may set the MPLLOCALFREETYPE environment variable to 1 to let Matplotlib download it."
error: Setup script exited with error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2

check is the matplotlib install error by default

(lbtest) E:\Github\labelme>conda install matplotlib
...
(lbtest) E:\Github\labelme>python setup.py build
running build
running build_py

(lbtest) E:\Github\labelme>python setup.py install
... # some crazy logs

Using c:\programdata\miniconda3\envs\lbtest\lib\site-packages
Finished processing dependencies for labelme==5.0.1

Then the labelme seems correctly installed.

(lbtest) E:\Github\labelme>labelme -h
usage: labelme [-h] [--version] [--reset-config] [--logger-level {debug,info,warning,fatal,error}] [--output OUTPUT] [--config CONFIG] [--nodata] [--autosave] [--nosortlabels] [--flags FLAGS] [--labelflags LABEL_FLAGS]
               [--labels LABELS] [--validatelabel {exact}] [--keep-prev] [--epsilon EPSILON]
               [filename]

positional arguments:
  filename              image or label filename

options:
  -h, --help            show this help message and exit
  --version, -V         show version
  --reset-config        reset qt config
  --logger-level {debug,info,warning,fatal,error}
                        logger level
  --output OUTPUT, -O OUTPUT, -o OUTPUT
                        output file or directory (if it ends with .json it is recognized as file, else as directory)
  --config CONFIG       config file or yaml-format string (default: C:\Users\CREST\.labelmerc)
  --nodata              stop storing image data to JSON file
  --autosave            auto save
  --nosortlabels        stop sorting labels
  --flags FLAGS         comma separated list of flags OR file containing flags
  --labelflags LABEL_FLAGS
                        yaml string of label specific flags OR file containing json string of label specific flags (ex. {person-\d+: [male, tall], dog-\d+: [black, brown, white], .*: [occluded]})
  --labels LABELS       comma separated list of labels OR file containing labels
  --validatelabel {exact}
                        label validation types
  --keep-prev           keep annotation of previous frame
  --epsilon EPSILON     epsilon to find nearest vertex on canvas

And run your tests, seems the GUI can not succesfully launch

(lbtest) E:\Github\labelme>labelme examples/semantic_segmentation/data_annotated/
This JSON file (examples/semantic_segmentation/data_annotated/2011_000003.json) may be incompatible with current labelme. version in file: 4.0.0, current version: 5.0.1
Traceback (most recent call last):
  File "C:\ProgramData\Miniconda3\envs\lbtest\lib\site-packages\labelme-5.0.1-py3.10.egg\labelme\app.py", line 1111, in fileSelectionChanged
    self.loadFile(filename)
  File "C:\ProgramData\Miniconda3\envs\lbtest\lib\site-packages\labelme-5.0.1-py3.10.egg\labelme\app.py", line 1538, in loadFile
    self.loadLabels(self.labelFile.shapes)
  File "C:\ProgramData\Miniconda3\envs\lbtest\lib\site-packages\labelme-5.0.1-py3.10.egg\labelme\app.py", line 1227, in loadLabels
    self.loadShapes(s)
  File "C:\ProgramData\Miniconda3\envs\lbtest\lib\site-packages\labelme-5.0.1-py3.10.egg\labelme\app.py", line 1188, in loadShapes
    self.addLabel(shape)
  File "C:\ProgramData\Miniconda3\envs\lbtest\lib\site-packages\labelme-5.0.1-py3.10.egg\labelme\app.py", line 1138, in addLabel
    self.labelList.addItem(label_list_item)
  File "C:\ProgramData\Miniconda3\envs\lbtest\lib\site-packages\labelme-5.0.1-py3.10.egg\labelme\widgets\label_list_widget.py", line 167, in addItem
    item.setSizeHint(self.itemDelegate().sizeHint(None, None))
  File "C:\ProgramData\Miniconda3\envs\lbtest\lib\site-packages\labelme-5.0.1-py3.10.egg\labelme\widgets\label_list_widget.py", line 64, in sizeHint
    return QtCore.QSize(
TypeError: arguments did not match any overloaded call:
  QSize(): too many arguments
  QSize(int, int): argument 1 has unexpected type 'float'
  QSize(QSize): argument 1 has unexpected type 'float'

wkentaro · 2022-06-28T13:19:45Z

The last error seems caused by Python 3.10.
#1020 (comment)
It will be fixed, but in the meanwhile, can you use 3.9 or older?

HowcanoeWang · 2022-06-29T02:42:30Z

Successfully on Python 3.7

The file list works for your examples (which json and img put together).

However, for our applications, which json and image at different folders, the problem still exists.

Then I checked the code in the app.py#L2037. This is caused by using the existed image name as list rather than json name as list.

Hoping it can add another logic in self.scanAllImages:

if img list is empty and json file exists:
    read the img path from json.image_path.
    if can find image from json.image_path
        append json filename to image list.

PS: Another UI issue -> #1142, it seems the line space in the "label list" panel is not correct in the high-resolution display. (May forgot to update its code? The "polygon labels" and "file list" seems to have correct line space)

wkentaro · 2022-06-29T04:47:19Z

Can you try —output option to specify the JSON directory? Can you open an issue for the panel issue?

HowcanoeWang · 2022-06-29T10:20:07Z

It does not work (because scanAllImages() filters image filename to processing list first, folders with only json labels will return an empty processing list).

I have fixed that by myself by editing only importDirImages and scanAllImages two functions: UTokyo-FieldPhenomics-Lab@1a9e8c6

However, when trying to pass all the tests before pull request, it seems in tests/data folder, there are many symbollink pin to other files:

When I clone to local, I could not access them, so some open image tests can not pass. Since I only modified both functions, personally think it is okay to pull request?

wkentaro · 2022-07-03T06:23:15Z

It does not work (because scanAllImages() filters image filename to processing list first, folders with only json labels will return an empty processing list).

You also need to specify the directory of images. something like below:

labelme JpegImageFiles/ --output AnnotatedJsonFiles/

HowcanoeWang · 2022-07-03T14:55:27Z

You also need to specify the directory of images. something like below:
labelme JpegImageFiles/ --output AnnotatedJsonFiles/

Thank you for sharing this point, this works but not so convenient.

I have modified the source code, now it can open the folder with only json directly (it will read the json file to get the directory of images), please check whether it is proper to merge into your main branch.

Thanks again for your patient reply!

wkentaro · 2022-09-26T14:50:06Z

By design, this cannot be achieved by Open dir, so we need something different Open annotated dir or something.

JAYCHOU2020 · 2023-10-27T07:12:30Z

By design, this cannot be achieved by Open dir, so we need something different Open annotated dir or something.

Thank you to the developers for their efforts. Has this function been implemented? Very necessary!

JAYCHOU2020 · 2023-10-27T08:26:35Z

I have modified the source code, now it can open the folder with only json directly (it will read the json file to get the directory of images), please check whether it is proper to merge into your main branch.

Thank you very much. I really, really need this feature you developed. I hope you can share the modified and optimized code with me. Thank you very much.

HowcanoeWang · 2023-10-27T09:20:04Z

I have modified the source code, now it can open the folder with only json directly (it will read the json file to get the directory of images), please check whether it is proper to merge into your main branch.

Thank you very much. I really, really need this feature you developed. I hope you can share the modified and optimized code with me. Thank you very much.

Kindly find it here: #1143, that pull request was rejected

But it was an old version (2 years ago?) if you use that directly.

HowcanoeWang changed the title ~~Open json file & json folder, file list panel is empty~~ [bug] Open json file & json folder, file list panel is empty Apr 7, 2022

wkentaro added the issue::bug label Jun 25, 2022

HowcanoeWang mentioned this issue Jun 30, 2022

Read directory with only json labels #1143

Open

wkentaro mentioned this issue Sep 26, 2022

[BUG] Open Dir not working correctly. #882

Closed

wkentaro added enhancement priority: low Low priority and removed issue::bug labels Sep 26, 2022

wkentaro removed the pr::enhancement label Dec 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bug] Open json file & json folder, file list panel is empty #1009

[bug] Open json file & json folder, file list panel is empty #1009

HowcanoeWang commented Apr 7, 2022

wkentaro commented Jun 25, 2022

HowcanoeWang commented Jun 27, 2022

wkentaro commented Jun 27, 2022

HowcanoeWang commented Jun 27, 2022

wkentaro commented Jun 27, 2022

HowcanoeWang commented Jun 27, 2022 •

edited

Loading

HowcanoeWang commented Jun 27, 2022 •

edited

Loading

wkentaro commented Jun 28, 2022

HowcanoeWang commented Jun 29, 2022 •

edited

Loading

wkentaro commented Jun 29, 2022 via email •

edited

Loading

HowcanoeWang commented Jun 29, 2022 •

edited

Loading

wkentaro commented Jul 3, 2022

HowcanoeWang commented Jul 3, 2022

wkentaro commented Sep 26, 2022

JAYCHOU2020 commented Oct 27, 2023

JAYCHOU2020 commented Oct 27, 2023

HowcanoeWang commented Oct 27, 2023 •

edited

Loading

[bug] Open json file & json folder, file list panel is empty #1009

[bug] Open json file & json folder, file list panel is empty #1009

Comments

HowcanoeWang commented Apr 7, 2022

wkentaro commented Jun 25, 2022

HowcanoeWang commented Jun 27, 2022

wkentaro commented Jun 27, 2022

HowcanoeWang commented Jun 27, 2022

wkentaro commented Jun 27, 2022

HowcanoeWang commented Jun 27, 2022 • edited Loading

create new conda env [failed]

install qt by conda [failed]

seems pyqt5 problem [solution not found]

HowcanoeWang commented Jun 27, 2022 • edited Loading

wkentaro commented Jun 28, 2022

HowcanoeWang commented Jun 29, 2022 • edited Loading

wkentaro commented Jun 29, 2022 via email • edited Loading

HowcanoeWang commented Jun 29, 2022 • edited Loading

wkentaro commented Jul 3, 2022

HowcanoeWang commented Jul 3, 2022

wkentaro commented Sep 26, 2022

JAYCHOU2020 commented Oct 27, 2023

JAYCHOU2020 commented Oct 27, 2023

HowcanoeWang commented Oct 27, 2023 • edited Loading

HowcanoeWang commented Jun 27, 2022 •

edited

Loading

HowcanoeWang commented Jun 27, 2022 •

edited

Loading

HowcanoeWang commented Jun 29, 2022 •

edited

Loading

wkentaro commented Jun 29, 2022 via email •

edited

Loading

HowcanoeWang commented Jun 29, 2022 •

edited

Loading

HowcanoeWang commented Oct 27, 2023 •

edited

Loading