fitz_new does not work with python3.10 or earlier

[paternal]

Describe the bug (mandatory)

fitz_new does not work with python3.10 or earlier

To Reproduce (mandatory)

# python3.10 -m pip install pymupdf && python3.10 -c "import fitz_new as fitz"
Requirement already satisfied: pymupdf in /usr/local/lib/python3.10/site-packages (1.23.5)
Requirement already satisfied: PyMuPDFb==1.23.5 in /usr/local/lib/python3.10/site-packages (from pymupdf) (1.23.5)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/fitz_new/__init__.py", line 130, in <module>
    from . import mupdf
  File "/usr/local/lib/python3.10/site-packages/fitz_new/mupdf.py", line 10, in <module>
    from . import _mupdf
ImportError: libcrypt.so.2: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.10/site-packages/fitz_new/__init__.py", line 132, in <module>
    import mupdf
ModuleNotFoundError: No module named 'mupdf'

We get the same result with python3.8 and python3.9.

Expected behavior (optional)

Here is what is happening with python3.11 or python3.12. I expect the same thing to happen with python3.8, python3.9 and python3.10.

# python3.11 -m pip install pymupdf && python3.11 -c "import fitz_new as fitz"
Requirement already satisfied: pymupdf in /usr/local/lib/python3.11/site-packages (1.23.5)
Requirement already satisfied: PyMuPDFb==1.23.5 in /usr/local/lib/python3.11/site-packages (from pymupdf) (1.23.5)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
platform/c++/implementation/internal.cpp:205:reinit_singlethreaded(): Reinitialising as single-threaded.

Your configuration (mandatory)

• Operating system, potentially version and bitness:
• Python version, bitness :
• PyMuPDF version, installation method (wheel or generated from source).

>>> print(sys.version, "\n", sys.platform, "\n", fitz.__doc__)
3.12.0 (main, Oct 14 2023, 07:51:24) [GCC 12.2.0] 
 linux 
 
PyMuPDF 1.23.5: Python bindings for the MuPDF 1.23.4 library.
Version date: 2023-10-11 00:00:01.
Built for Python 3.12 on linux (64-bit).

PyMuPDF has been installed using pip.

Additional context (optional)

I do not need it. I am just reporting it to inform you, if you do not know it yet.

I need to use fitz_new because of #2742. A workaround is to import PyMuPDF with:

if sys.version_info >= (3, 11):
    import fitz_new as fitz
else:
    import fitz

Thanks for your work!

[JorjMcKie]

Thanks for reporting this!

This is a known problem of Python versions < 3.11.
There exists a formal, but unneeded dependency for that libcrypt.so.2 in these Python versions.
Has been removed thereafter.
Grotesque as it may seem, when adding a (renamed) copy of any existing libcrypt.so in the respective folder, all will be roses again.

[JorjMcKie]

Example circumvention:

python3
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import fitz_new
Traceback (most recent call last):
  File "/home/.../fitz_new/__init__.py", line 130, in <module>
    from . import mupdf
  File "/home/.../fitz_new/mupdf.py", line 10, in <module>
    from . import _mupdf
ImportError: libcrypt.so.2: cannot open shared object file: No such file or directory
...
>>> exit()
whereis libcrypt.so.1
libcrypt.so.1: /usr/lib/x86_64-linux-gnu/libcrypt.so.1
cd /usr/lib/x86_64-linux-gnu
sudo cp libcrypt.so.1 libcrypt.so.2
python3
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import fitz_new
platform/c++/implementation/internal.cpp:205:reinit_singlethreaded(): Reinitialising as single-threaded.
>>> # EVERYTHING WORKS!

[paternal]

Thanks for your quick reply!

I am hijacking my own thread to ask: the changelog reads "[fitz_new] can be used as a drop-in replacement with import fitz_new as fitz." I found one case where it is not the case (i.e. using fitz_new instead of fitz breaks something). Should I report an issue, or should I find a workaround and stop bothering you?

[JorjMcKie]

Thanks for your quick reply!

I am hijacking my own thread to ask: the changelog reads "[fitz_new] can be used as a drop-in replacement with import fitz_new as fitz." I found one case where it is not the case (i.e. using fitz_new instead of fitz breaks something). Should I report an issue, or should I find a workaround and stop bothering you?

That would be a valuable finding and definitely no bothering!
Please do report as many issues as you may find!👌👍

[julian-smith-artifex-com]

Actually we have a fix for this problem on older Python versions (it's an incorrect -lcrypt flag from python-config). Unfortunately the fix it did not make it into the latest MuPDF release, and this means it still effects the rebased implementation of PyMuPDF.

So this problem is understood, and will be fixed, hopefully in the next PyMuPDF release, depending on the MuPDF release schedule.

[julian-smith-artifex-com]

Fixed in 1.23.6.