-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
nbval and cov do not play nicely together #116
Comments
Full output:
|
I am getting this same error as well. I tried downgrading notebook from |
@saulshanabrook Are you getting the same error with the same notebooks, or with different ones? |
@vidartf Different notebooks. In https://github.com/quansight-labs/metadsl |
Are either one of you able to produce a minimal working example? Its hard to debug behavior in a completely unknown repository. |
I was able to reproduce this in an minimal conda environment created like this: conda create -n tmp -c conda-forge jupyterlab
conda activate tmp
pip install pytest pytest-cov nbval
# Create notebook file `Untitled.ipynb`
pytest --cov --nbval Untitled.ipynb Output:========================================================= test session starts =========================================================
platform darwin -- Python 3.7.3, pytest-4.5.0, py-1.8.0, pluggy-0.11.0
rootdir: /Users/saul/p/tmp
plugins: cov-2.7.1, nbval-0.9.1
collected 2 items
Untitled.ipynb EE [100%]Coverage.py warning: No data was collected. (no-data-collected)
WARNING: Failed to generate report: No data to report.
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pytest_cov/plugin.py:229: PytestWarning: Failed to generate report: No data to report.
self.cov_controller.finish()
=============================================================== ERRORS ================================================================
______________________________________________ ERROR at setup of Untitled.ipynb::Cell 0 _______________________________________________
cls = <class '_pytest.runner.CallInfo'>, func = <function call_runtest_hook.<locals>.<lambda> at 0x10a1ed730>, when = 'setup'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)
@classmethod
def from_call(cls, func, when, reraise=None):
#: context of invocation: one of "setup", "call",
#: "teardown", "memocollect"
start = time()
excinfo = None
try:
> result = func()
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/_pytest/runner.py:225:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> lambda: ihook(item=item, **kwds), when=when, reraise=reraise
)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/_pytest/runner.py:197:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <_HookCaller 'pytest_runtest_setup'>, args = (), kwargs = {'item': <IPyNbCell Cell 0>}, notincall = set()
def __call__(self, *args, **kwargs):
if args:
raise TypeError("hook calling supports only keyword arguments")
assert not self.is_historic()
if self.spec and self.spec.argnames:
notincall = (
set(self.spec.argnames) - set(["__multicall__"]) - set(kwargs.keys())
)
if notincall:
warnings.warn(
"Argument(s) {} which are declared in the hookspec "
"can not be found in this hook call".format(tuple(notincall)),
stacklevel=2,
)
> return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/hooks.py:289:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <_pytest.config.PytestPluginManager object at 0x106ab4128>, hook = <_HookCaller 'pytest_runtest_setup'>
methods = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from '/usr/local/miniconda3/envs/tmp/lib/python3.7/site-p...apture 0 oldfd=5 _state='started'> _state='suspended' _in_suspended='<UNSET>'> _current_item=<IPyNbCell Cell 0>>>, ...]
kwargs = {'item': <IPyNbCell Cell 0>}
def _hookexec(self, hook, methods, kwargs):
# called from all hookcaller instances.
# enable_tracing will set its own wrapping function at self._inner_hookexec
> return self._inner_hookexec(hook, methods, kwargs)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/manager.py:68:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hook = <_HookCaller 'pytest_runtest_setup'>
methods = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from '/usr/local/miniconda3/envs/tmp/lib/python3.7/site-p...apture 0 oldfd=5 _state='started'> _state='suspended' _in_suspended='<UNSET>'> _current_item=<IPyNbCell Cell 0>>>, ...]
kwargs = {'item': <IPyNbCell Cell 0>}
self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
methods,
kwargs,
> firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/manager.py:62:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hook_impls = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from '/usr/local/miniconda3/envs/tmp/lib/python3.7/site-p...apture 0 oldfd=5 _state='started'> _state='suspended' _in_suspended='<UNSET>'> _current_item=<IPyNbCell Cell 0>>>, ...]
caller_kwargs = {'item': <IPyNbCell Cell 0>}, firstresult = False
def _multicall(hook_impls, caller_kwargs, firstresult=False):
"""Execute a call into multiple python functions/methods and return the
result(s).
``caller_kwargs`` comes from _HookCaller.__call__().
"""
__tracebackhide__ = True
results = []
excinfo = None
try: # run impl and wrapper setup functions in a loop
teardowns = []
try:
for hook_impl in reversed(hook_impls):
try:
args = [caller_kwargs[argname] for argname in hook_impl.argnames]
except KeyError:
for argname in hook_impl.argnames:
if argname not in caller_kwargs:
raise HookCallError(
"hook call must provide argument %r" % (argname,)
)
if hook_impl.hookwrapper:
try:
gen = hook_impl.function(*args)
next(gen) # first yield
teardowns.append(gen)
except StopIteration:
_raise_wrapfail(gen, "did not yield")
else:
res = hook_impl.function(*args)
if res is not None:
results.append(res)
if firstresult: # halt further impl calls
break
except BaseException:
excinfo = sys.exc_info()
finally:
if firstresult: # first result hooks return a single value
outcome = _Result(results[0] if results else None, excinfo)
else:
outcome = _Result(results, excinfo)
# run all wrapper post-yield blocks
for gen in reversed(teardowns):
try:
gen.send(outcome)
_raise_wrapfail(gen, "has second yield")
except StopIteration:
pass
> return outcome.get_result()
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/callers.py:208:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pluggy.callers._Result object at 0x10a22de10>
def get_result(self):
"""Get the result(s) for this hook call.
If the hook was marked as a ``firstresult`` only a single value
will be returned otherwise a list of results.
"""
__tracebackhide__ = True
if self._excinfo is None:
return self._result
else:
ex = self._excinfo
if _py3:
> raise ex[1].with_traceback(ex[2])
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/callers.py:80:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hook_impls = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from '/usr/local/miniconda3/envs/tmp/lib/python3.7/site-p...apture 0 oldfd=5 _state='started'> _state='suspended' _in_suspended='<UNSET>'> _current_item=<IPyNbCell Cell 0>>>, ...]
caller_kwargs = {'item': <IPyNbCell Cell 0>}, firstresult = False
def _multicall(hook_impls, caller_kwargs, firstresult=False):
"""Execute a call into multiple python functions/methods and return the
result(s).
``caller_kwargs`` comes from _HookCaller.__call__().
"""
__tracebackhide__ = True
results = []
excinfo = None
try: # run impl and wrapper setup functions in a loop
teardowns = []
try:
for hook_impl in reversed(hook_impls):
try:
args = [caller_kwargs[argname] for argname in hook_impl.argnames]
except KeyError:
for argname in hook_impl.argnames:
if argname not in caller_kwargs:
raise HookCallError(
"hook call must provide argument %r" % (argname,)
)
if hook_impl.hookwrapper:
try:
gen = hook_impl.function(*args)
next(gen) # first yield
teardowns.append(gen)
except StopIteration:
_raise_wrapfail(gen, "did not yield")
else:
> res = hook_impl.function(*args)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/callers.py:187:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
item = <IPyNbCell Cell 0>
def pytest_runtest_setup(item):
_update_current_test_var(item, "setup")
> item.session._setupstate.prepare(item)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/_pytest/runner.py:115:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <_pytest.runner.SetupState object at 0x10a1e8f28>, colitem = <IPyNbCell Cell 0>
def prepare(self, colitem):
""" setup objects along the collector chain to the test-method
and teardown previously setup objects."""
needed_collectors = colitem.listchain()
self._teardown_towards(needed_collectors)
# check if the last collection node has raised an error
for col in self.stack:
if hasattr(col, "_prepare_exc"):
six.reraise(*col._prepare_exc)
for col in needed_collectors[len(self.stack) :]:
self.stack.append(col)
try:
> col.setup()
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/_pytest/runner.py:361:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <IPyNbFile Untitled.ipynb>
def setup(self):
"""
Called by pytest to setup the collector cells in .
Here we start a kernel and setup the sanitize patterns.
"""
if self.parent.config.option.current_env:
kernel_name = CURRENT_ENV_KERNEL_NAME
else:
kernel_name = self.nb.metadata.get(
'kernelspec', {}).get('name', 'python')
self.kernel = RunningKernel(kernel_name, str(self.fspath.dirname))
self.setup_sanitize_files()
if getattr(self.parent.config.option, 'cov_source', None):
> setup_coverage(self.parent.config, self.kernel, getattr(self, "fspath", None))
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/nbval/plugin.py:235:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
config = <_pytest.config.Config object at 0x10794a0f0>, kernel = <nbval.kernel.RunningKernel object at 0x10a2372b0>
floc = local('/Users/saul/p/tmp/Untitled.ipynb'), output_loc = None
def setup_coverage(config, kernel, floc, output_loc=None):
"""Start coverage reporting in kernel.
Currently supported kernel languages are:
- Python
"""
language = kernel.language
if language.startswith('python'):
# Get the pytest-cov coverage object
cov = get_cov(config)
if cov:
# If present, copy the data file location used by pytest-cov
data_file = os.path.abspath(cov.config.data_file)
else:
# Fall back on output_loc and current dir if not
data_file = os.path.abspath(os.path.join(output_loc or os.getcwd(), '.coverage'))
# Get options from pytest-cov's command line arguments:
source = config.option.cov_source
config_file = config.option.cov_config
if isinstance(config_file, str) and os.path.isfile(config_file):
config_file = os.path.abspath(config_file)
# Copy the suffix of plugin if available
suffix = _make_suffix(cov)
if suffix is True:
# Cannot merge data with autogen suffix, so turn off warning
# for missing data in pytest-cov collector
cov._warn_no_data = False
# Build setup command and execute in kernel:
cmd = _python_setup % (data_file, source, config_file, suffix)
msg_id = kernel.kc.execute(cmd, stop_on_error=False)
> kernel.await_idle(msg_id, 60) # A minute should be plenty to enable coverage
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/nbval/cover.py:67:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <nbval.kernel.RunningKernel object at 0x10a2372b0>, parent_id = '38a50b57-0eb629018ade829489b5c38f', timeout = 60
def await_idle(self, parent_id, timeout):
"""Poll the iopub stream until an idle message is received for the given parent ID"""
while True:
# Get a message from the kernel iopub channel
> msg = self.get_message(timeout=timeout, stream='iopub') # raises Empty on timeout!
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/nbval/kernel.py:170:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <nbval.kernel.RunningKernel object at 0x10a2372b0>, stream = 'iopub', timeout = 60
def get_message(self, stream, timeout=None):
"""
Function is used to get a message from the iopub channel.
Timeout is None by default
When timeout is reached
"""
try:
if stream == 'iopub':
> msg = self.kc.get_iopub_msg(timeout=timeout)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/nbval/kernel.py:123:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <jupyter_client.blocking.client.BlockingKernelClient object at 0x10a7398d0>, args = (), kwargs = {'timeout': 60}
def get_iopub_msg(self, *args, **kwargs):
"""Get a message from the iopub channel"""
> return self.iopub_channel.get_msg(*args, **kwargs)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/jupyter_client/client.py:81:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <jupyter_client.blocking.channels.ZMQSocketChannel object at 0x10a739a90>, block = True, timeout = 60000
def get_msg(self, block=True, timeout=None):
""" Gets a message if there is one that is ready. """
if block:
if timeout is not None:
timeout *= 1000 # seconds to ms
ready = self.socket.poll(timeout)
else:
ready = self.socket.poll(timeout=0)
if ready:
return self._recv()
else:
> raise Empty
E _queue.Empty
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/jupyter_client/blocking/channels.py:57: Empty
______________________________________________ ERROR at setup of Untitled.ipynb::Cell 1 _______________________________________________
cls = <class '_pytest.runner.CallInfo'>, func = <function call_runtest_hook.<locals>.<lambda> at 0x10a21ba60>, when = 'setup'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)
@classmethod
def from_call(cls, func, when, reraise=None):
#: context of invocation: one of "setup", "call",
#: "teardown", "memocollect"
start = time()
excinfo = None
try:
> result = func()
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/_pytest/runner.py:225:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> lambda: ihook(item=item, **kwds), when=when, reraise=reraise
)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/_pytest/runner.py:197:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <_HookCaller 'pytest_runtest_setup'>, args = (), kwargs = {'item': <IPyNbCell Cell 1>}, notincall = set()
def __call__(self, *args, **kwargs):
if args:
raise TypeError("hook calling supports only keyword arguments")
assert not self.is_historic()
if self.spec and self.spec.argnames:
notincall = (
set(self.spec.argnames) - set(["__multicall__"]) - set(kwargs.keys())
)
if notincall:
warnings.warn(
"Argument(s) {} which are declared in the hookspec "
"can not be found in this hook call".format(tuple(notincall)),
stacklevel=2,
)
> return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/hooks.py:289:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <_pytest.config.PytestPluginManager object at 0x106ab4128>, hook = <_HookCaller 'pytest_runtest_setup'>
methods = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from '/usr/local/miniconda3/envs/tmp/lib/python3.7/site-p...apture 0 oldfd=5 _state='started'> _state='suspended' _in_suspended='<UNSET>'> _current_item=<IPyNbCell Cell 1>>>, ...]
kwargs = {'item': <IPyNbCell Cell 1>}
def _hookexec(self, hook, methods, kwargs):
# called from all hookcaller instances.
# enable_tracing will set its own wrapping function at self._inner_hookexec
> return self._inner_hookexec(hook, methods, kwargs)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/manager.py:68:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hook = <_HookCaller 'pytest_runtest_setup'>
methods = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from '/usr/local/miniconda3/envs/tmp/lib/python3.7/site-p...apture 0 oldfd=5 _state='started'> _state='suspended' _in_suspended='<UNSET>'> _current_item=<IPyNbCell Cell 1>>>, ...]
kwargs = {'item': <IPyNbCell Cell 1>}
self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
methods,
kwargs,
> firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/manager.py:62:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hook_impls = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from '/usr/local/miniconda3/envs/tmp/lib/python3.7/site-p...apture 0 oldfd=5 _state='started'> _state='suspended' _in_suspended='<UNSET>'> _current_item=<IPyNbCell Cell 1>>>, ...]
caller_kwargs = {'item': <IPyNbCell Cell 1>}, firstresult = False
def _multicall(hook_impls, caller_kwargs, firstresult=False):
"""Execute a call into multiple python functions/methods and return the
result(s).
``caller_kwargs`` comes from _HookCaller.__call__().
"""
__tracebackhide__ = True
results = []
excinfo = None
try: # run impl and wrapper setup functions in a loop
teardowns = []
try:
for hook_impl in reversed(hook_impls):
try:
args = [caller_kwargs[argname] for argname in hook_impl.argnames]
except KeyError:
for argname in hook_impl.argnames:
if argname not in caller_kwargs:
raise HookCallError(
"hook call must provide argument %r" % (argname,)
)
if hook_impl.hookwrapper:
try:
gen = hook_impl.function(*args)
next(gen) # first yield
teardowns.append(gen)
except StopIteration:
_raise_wrapfail(gen, "did not yield")
else:
res = hook_impl.function(*args)
if res is not None:
results.append(res)
if firstresult: # halt further impl calls
break
except BaseException:
excinfo = sys.exc_info()
finally:
if firstresult: # first result hooks return a single value
outcome = _Result(results[0] if results else None, excinfo)
else:
outcome = _Result(results, excinfo)
# run all wrapper post-yield blocks
for gen in reversed(teardowns):
try:
gen.send(outcome)
_raise_wrapfail(gen, "has second yield")
except StopIteration:
pass
> return outcome.get_result()
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/callers.py:208:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pluggy.callers._Result object at 0x10a8e39e8>
def get_result(self):
"""Get the result(s) for this hook call.
If the hook was marked as a ``firstresult`` only a single value
will be returned otherwise a list of results.
"""
__tracebackhide__ = True
if self._excinfo is None:
return self._result
else:
ex = self._excinfo
if _py3:
> raise ex[1].with_traceback(ex[2])
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/callers.py:80:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hook_impls = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from '/usr/local/miniconda3/envs/tmp/lib/python3.7/site-p...apture 0 oldfd=5 _state='started'> _state='suspended' _in_suspended='<UNSET>'> _current_item=<IPyNbCell Cell 1>>>, ...]
caller_kwargs = {'item': <IPyNbCell Cell 1>}, firstresult = False
def _multicall(hook_impls, caller_kwargs, firstresult=False):
"""Execute a call into multiple python functions/methods and return the
result(s).
``caller_kwargs`` comes from _HookCaller.__call__().
"""
__tracebackhide__ = True
results = []
excinfo = None
try: # run impl and wrapper setup functions in a loop
teardowns = []
try:
for hook_impl in reversed(hook_impls):
try:
args = [caller_kwargs[argname] for argname in hook_impl.argnames]
except KeyError:
for argname in hook_impl.argnames:
if argname not in caller_kwargs:
raise HookCallError(
"hook call must provide argument %r" % (argname,)
)
if hook_impl.hookwrapper:
try:
gen = hook_impl.function(*args)
next(gen) # first yield
teardowns.append(gen)
except StopIteration:
_raise_wrapfail(gen, "did not yield")
else:
> res = hook_impl.function(*args)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/pluggy/callers.py:187:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
item = <IPyNbCell Cell 1>
def pytest_runtest_setup(item):
_update_current_test_var(item, "setup")
> item.session._setupstate.prepare(item)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/_pytest/runner.py:115:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <_pytest.runner.SetupState object at 0x10a1e8f28>, colitem = <IPyNbCell Cell 1>
def prepare(self, colitem):
""" setup objects along the collector chain to the test-method
and teardown previously setup objects."""
needed_collectors = colitem.listchain()
self._teardown_towards(needed_collectors)
# check if the last collection node has raised an error
for col in self.stack:
if hasattr(col, "_prepare_exc"):
> six.reraise(*col._prepare_exc)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/_pytest/runner.py:357:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tp = <class '_queue.Empty'>, value = None, tb = None
def reraise(tp, value, tb=None):
try:
if value is None:
value = tp()
if value.__traceback__ is not tb:
> raise value.with_traceback(tb)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/six.py:692:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <_pytest.runner.SetupState object at 0x10a1e8f28>, colitem = <IPyNbCell Cell 0>
def prepare(self, colitem):
""" setup objects along the collector chain to the test-method
and teardown previously setup objects."""
needed_collectors = colitem.listchain()
self._teardown_towards(needed_collectors)
# check if the last collection node has raised an error
for col in self.stack:
if hasattr(col, "_prepare_exc"):
six.reraise(*col._prepare_exc)
for col in needed_collectors[len(self.stack) :]:
self.stack.append(col)
try:
> col.setup()
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/_pytest/runner.py:361:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <IPyNbFile Untitled.ipynb>
def setup(self):
"""
Called by pytest to setup the collector cells in .
Here we start a kernel and setup the sanitize patterns.
"""
if self.parent.config.option.current_env:
kernel_name = CURRENT_ENV_KERNEL_NAME
else:
kernel_name = self.nb.metadata.get(
'kernelspec', {}).get('name', 'python')
self.kernel = RunningKernel(kernel_name, str(self.fspath.dirname))
self.setup_sanitize_files()
if getattr(self.parent.config.option, 'cov_source', None):
> setup_coverage(self.parent.config, self.kernel, getattr(self, "fspath", None))
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/nbval/plugin.py:235:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
config = <_pytest.config.Config object at 0x10794a0f0>, kernel = <nbval.kernel.RunningKernel object at 0x10a2372b0>
floc = local('/Users/saul/p/tmp/Untitled.ipynb'), output_loc = None
def setup_coverage(config, kernel, floc, output_loc=None):
"""Start coverage reporting in kernel.
Currently supported kernel languages are:
- Python
"""
language = kernel.language
if language.startswith('python'):
# Get the pytest-cov coverage object
cov = get_cov(config)
if cov:
# If present, copy the data file location used by pytest-cov
data_file = os.path.abspath(cov.config.data_file)
else:
# Fall back on output_loc and current dir if not
data_file = os.path.abspath(os.path.join(output_loc or os.getcwd(), '.coverage'))
# Get options from pytest-cov's command line arguments:
source = config.option.cov_source
config_file = config.option.cov_config
if isinstance(config_file, str) and os.path.isfile(config_file):
config_file = os.path.abspath(config_file)
# Copy the suffix of plugin if available
suffix = _make_suffix(cov)
if suffix is True:
# Cannot merge data with autogen suffix, so turn off warning
# for missing data in pytest-cov collector
cov._warn_no_data = False
# Build setup command and execute in kernel:
cmd = _python_setup % (data_file, source, config_file, suffix)
msg_id = kernel.kc.execute(cmd, stop_on_error=False)
> kernel.await_idle(msg_id, 60) # A minute should be plenty to enable coverage
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/nbval/cover.py:67:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <nbval.kernel.RunningKernel object at 0x10a2372b0>, parent_id = '38a50b57-0eb629018ade829489b5c38f', timeout = 60
def await_idle(self, parent_id, timeout):
"""Poll the iopub stream until an idle message is received for the given parent ID"""
while True:
# Get a message from the kernel iopub channel
> msg = self.get_message(timeout=timeout, stream='iopub') # raises Empty on timeout!
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/nbval/kernel.py:170:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <nbval.kernel.RunningKernel object at 0x10a2372b0>, stream = 'iopub', timeout = 60
def get_message(self, stream, timeout=None):
"""
Function is used to get a message from the iopub channel.
Timeout is None by default
When timeout is reached
"""
try:
if stream == 'iopub':
> msg = self.kc.get_iopub_msg(timeout=timeout)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/nbval/kernel.py:123:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <jupyter_client.blocking.client.BlockingKernelClient object at 0x10a7398d0>, args = (), kwargs = {'timeout': 60}
def get_iopub_msg(self, *args, **kwargs):
"""Get a message from the iopub channel"""
> return self.iopub_channel.get_msg(*args, **kwargs)
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/jupyter_client/client.py:81:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <jupyter_client.blocking.channels.ZMQSocketChannel object at 0x10a739a90>, block = True, timeout = 60000
def get_msg(self, block=True, timeout=None):
""" Gets a message if there is one that is ready. """
if block:
if timeout is not None:
timeout *= 1000 # seconds to ms
ready = self.socket.poll(timeout)
else:
ready = self.socket.poll(timeout=0)
if ready:
return self._recv()
else:
> raise Empty
E _queue.Empty
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/jupyter_client/blocking/channels.py:57: Empty
========================================================== warnings summary ===========================================================
Untitled.ipynb::Cell 0
/usr/local/miniconda3/envs/tmp/lib/python3.7/site-packages/jupyter_client/manager.py:62: DeprecationWarning: KernelManager._kernel_spec_manager_changed is deprecated in traitlets 4.1: use @observe and @unobserve instead.
def _kernel_spec_manager_changed(self):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
---------- coverage: platform darwin, python 3.7.3-final-0 -----------
Name Stmts Miss Cover
---------------------------
================================================ 1 warnings, 2 error in 62.50 seconds ================================================= Environment:$ pip freeze
appnope==0.1.0
atomicwrites==1.3.0
attrs==19.1.0
backcall==0.1.0
bleach==3.1.0
certifi==2019.3.9
coverage==4.5.3
decorator==4.4.0
defusedxml==0.5.0
entrypoints==0.3
flake8==3.6.0
ipykernel==5.1.0
ipython==7.5.0
ipython-genutils==0.2.0
jedi==0.13.3
Jinja2==2.10.1
jsonschema==3.0.1
jupyter-client==5.2.4
jupyter-core==4.4.0
jupyterlab==0.35.6
jupyterlab-server==0.2.0
MarkupSafe==1.1.1
mccabe==0.6.1
mistune==0.8.4
more-itertools==7.0.0
mypy-extensions==0.4.1
nbconvert==5.5.0
nbformat==4.4.0
nbval==0.9.1
notebook==5.7.8
pandocfilters==1.4.2
parso==0.4.0
pexpect==4.7.0
pickleshare==0.7.5
pluggy==0.11.0
prometheus-client==0.6.0
prompt-toolkit==2.0.9
ptyprocess==0.6.0
py==1.8.0
pycodestyle==2.4.0
pyflakes==2.0.0
Pygments==2.4.0
pyrsistent==0.15.2
pytest==4.5.0
pytest-cov==2.7.1
python-dateutil==2.8.0
pyzmq==18.0.1
Send2Trash==1.5.0
six==1.12.0
terminado==0.8.2
testpath==0.4.2
tornado==6.0.2
traitlets==4.3.2
wcwidth==0.1.7
webencodings==0.5.1
$ conda info
active environment : tmp
active env location : /usr/local/miniconda3/envs/tmp
shell level : 1
user config file : /Users/saul/.condarc
populated config files :
conda version : 4.6.12
conda-build version : not installed
python version : 3.6.8.final.0
base environment : /usr/local/miniconda3 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/osx-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/osx-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /usr/local/miniconda3/pkgs
/Users/saul/.conda/pkgs
envs directories : /usr/local/miniconda3/envs
/Users/saul/.conda/envs
platform : osx-64
user-agent : conda/4.6.12 requests/2.18.4 CPython/3.6.8 Darwin/17.7.0 OSX/10.13.6
UID:GID : 501:20
netrc file : /Users/saul/.netrc
offline mode : False
$ cat Untitled.ipynb
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"1 + 1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
} |
Weird, I cannot reproduce. Diff from - appnope==0.1.0
+colorama==0.4.1
- mccabe==0.6.1
- mypy-extensions==0.4.1
- pexpect==4.7.0
- ptyprocess==0.6.0
- pycodestyle==2.4.0
- pyflakes==2.0.0
+wincertstore==0.2 The package diff is mostly platform-based (I assume). Speaking of platform, I see that both of you are on OSX. Are you able to reproduce this on other platforms? I'm running on Windows here. |
Anyone solved this issue yet or maybe found a workaround? |
Specifying a module to cover means it won't hang, but also it won't actually pick up coverage: pytest --cov=mymod --cov-report=term --nbval tests/test_mymod.ipynb
...
tests/test_mymod::ipynb::Cell 0 PASSED
Coverage.py warning: Module mymod was never imported. (module-not-imported)
Coverage.py warning: No data was collected. (no-data-collected)
WARNING: Failed to generate report: No data to report.
PytestWarning: Failed to generate report: No data to report.
... EDIT: this seems to be because Note that with normal python files ( |
Steps to reproduce:
uarray
: https://github.com/Quansight-Labs/uarray.conda/environment.yml
nbval
addopts
line inpytest.ini
pytest --cov --nbval .
Actual output
nbval stalls, then errors with a timeout error.
Expected output
nbval should test the notebooks without a timeout.
OS/environment:
macOS 10.14
conda list
The text was updated successfully, but these errors were encountered: