Doctests fail when pytest is run

[bcdarwin]

Description

doctests assume

import pycm
import numpy as np

Steps/Code to Reproduce

pytest

Actual Behavior

============================= test session starts ==============================
platform linux -- Python 3.10.10, pytest-7.2.0, pluggy-1.0.0
rootdir: /build/source, configfile: pytest.ini
collected 15 items                                                             

Test/compare_test.py .                                                   [  6%]
Test/curve_test.py .                                                     [ 13%]
Test/error_test.py .                                                     [ 20%]
Test/function_test.py .                                                  [ 26%]
Test/output_test.py .                                                    [ 33%]
Test/overall_test.py .                                                   [ 40%]
Test/plot_test.py .                                                      [ 46%]
Test/verified_test.py .                                                  [ 53%]
Test/warning_test.py .                                                   [ 60%]
pycm/pycm_compare.py F                                                   [ 66%]
pycm/pycm_curve.py FFF                                                   [ 86%]
pycm/pycm_distance.py F                                                  [ 93%]
pycm/pycm_obj.py .                                                       [100%]

=================================== FAILURES ===================================
_____________________ [doctest] pycm.pycm_compare.Compare ______________________
015 
016     Compare class.
017 
018     >>> cm1 = ConfusionMatrix(matrix={0:{0:2,1:50,2:6},1:{0:5,1:50,2:3},2:{0:1,1:7,2:50}})
019     >>> cm2 = ConfusionMatrix(matrix={0:{0:50,1:2,2:6},1:{0:50,1:5,2:3},2:{0:1,1:55,2:2}})
020     >>> cp = Compare({"cm1":cm1,"cm2":cm2})
021     >>> print(cp)
Expected:
    Best : cm1
Got:
    Best : cm1
    <BLANKLINE>
    Rank  Name   Class-Score       Overall-Score
    1     cm1    0.50278           0.58095
    2     cm2    0.33611           0.52857
    <BLANKLINE>

/build/source/pycm/pycm_compare.py:21: DocTestFailure
_______________________ [doctest] pycm.pycm_curve.Curve ________________________
013 
014     Curve class.
015 
016     >>> crv = Curve(actual_vector = np.array([1, 1, 2, 2]), probs = np.array([[0.1, 0.9], [0.4, 0.6], [0.35, 0.65], [0.8, 0.2]]), classes=[2, 1])
UNEXPECTED EXCEPTION: NameError("name 'np' is not defined")
Traceback (most recent call last):
  File "/nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/lib/python3.10/doctest.py", line 1350, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest pycm.pycm_curve.Curve[0]>", line 1, in <module>
NameError: name 'np' is not defined
/build/source/pycm/pycm_curve.py:16: UnexpectedException
______________________ [doctest] pycm.pycm_curve.PRCurve _______________________
209 
210     PRCurve class.
211 
212     >>> crv = PRCurve(actual_vector = np.array([1, 1, 2, 2]), probs = np.array([[0.1, 0.9], [0.4, 0.6], [0.35, 0.65], [0.8, 0.2]]), classes=[2, 1])
UNEXPECTED EXCEPTION: NameError("name 'np' is not defined")
Traceback (most recent call last):
  File "/nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/lib/python3.10/doctest.py", line 1350, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest pycm.pycm_curve.PRCurve[0]>", line 1, in <module>
NameError: name 'np' is not defined
/build/source/pycm/pycm_curve.py:212: UnexpectedException
______________________ [doctest] pycm.pycm_curve.ROCCurve ______________________
168 
169     ROCCurve class.
170 
171     >>> crv = ROCCurve(actual_vector = np.array([1, 1, 2, 2]), probs = np.array([[0.1, 0.9], [0.4, 0.6], [0.35, 0.65], [0.8, 0.2]]), classes=[2, 1])
UNEXPECTED EXCEPTION: NameError("name 'np' is not defined")
Traceback (most recent call last):
  File "/nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/lib/python3.10/doctest.py", line 1350, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest pycm.pycm_curve.ROCCurve[0]>", line 1, in <module>
NameError: name 'np' is not defined
/build/source/pycm/pycm_curve.py:171: UnexpectedException
__________________ [doctest] pycm.pycm_distance.DistanceType ___________________
009 
010     Distance metric type class.
011 
012     >>> pycm.DistanceType.AMPLE
UNEXPECTED EXCEPTION: NameError("name 'pycm' is not defined")
Traceback (most recent call last):
  File "/nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/lib/python3.10/doctest.py", line 1350, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest pycm.pycm_distance.DistanceType[0]>", line 1, in <module>
NameError: name 'pycm' is not defined
/build/source/pycm/pycm_distance.py:12: UnexpectedException
=============================== warnings summary ===============================
pycm/pycm_compare.py:87
  /build/source/pycm/pycm_compare.py:87: RuntimeWarning: Confusion matrices are too close and the best one can not be recognized.
    warn(COMPARE_RESULT_WARNING, RuntimeWarning)

Test/curve_test.py::curve_test
Test/verified_test.py::verified_test
  /build/source/pycm/pycm_curve.py:379: RuntimeWarning: The curve axes contain non-numerical value(s).
    warn(CURVE_NONE_WARNING, RuntimeWarning)

Test/function_test.py::function_test
Test/function_test.py::function_test
  /nix/store/6m46482pl9qxld9xbnc9kkkqsf2a9kqs-python3.10-numpy-1.23.5/lib/python3.10/site-packages/numpy/core/_methods.py:48: DeprecationWarning: The `dtype` and `signature` arguments to ufuncs only select the general DType and not details such as the byte order or time unit (with rare exceptions see release notes).  To avoid this warning please use the scalar types `np.float64`, or string notation.
  In rare cases where the time unit was preserved, either cast the inputs or provide an output array. In the future NumPy may transition to allow providing `dtype=` to denote the outputs `dtype` as well. (Deprecated NumPy 1.21)
    return umr_sum(a, axis, dtype, out, keepdims, initial, where)

Test/plot_test.py::plot_test
  /build/source/pycm/pycm_curve.py:302: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`). Consider using `matplotlib.pyplot.close()`.
    fig, ax = plt.subplots()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED pycm/pycm_compare.py::pycm.pycm_compare.Compare
FAILED pycm/pycm_curve.py::pycm.pycm_curve.Curve
FAILED pycm/pycm_curve.py::pycm.pycm_curve.PRCurve
FAILED pycm/pycm_curve.py::pycm.pycm_curve.ROCCurve
FAILED pycm/pycm_distance.py::pycm.pycm_distance.DistanceType
=================== 5 failed, 10 passed, 6 warnings in 5.52s ===================

Python Version

3.10

PyCM Version (Use : pycm.__version__)

3.7 and 3.8

[sadrasabouri]

Hi @bcdarwin,
Thanks for your issue. Please check out here for the correct use of the pytest in this repository.

If your question is still valid update this issue thread, close it otherwise.

[bcdarwin]

Thanks for your reply.

I was able to run the tests (by passing "Test" to pytest), but the main point of the issue is that the doctests don't work. If this is acceptable to you, feel free to close. In this case it might also be good to document in the readme or configure pytest appropriately.

[sadrasabouri]

Aha, I see.
What do you think guys?
@sepandhaghighi @alirezazolanvari