Configure pytest #1268
Closed
Configure pytest #1268
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This section appears to be left over. It is only necessary in combination with coverage combine, which we are not using.
1 similar comment
|
One thing that I like to do is to include the tests folder in coverage. This does skew the overall coverage number but it does help to spot parts of the tests that are not executed. |
| - python setup.py install | ||
| - flake8 --exit-zero rdflib | ||
| - PYTHONWARNINGS=default nosetests --with-timer --timer-top-n 42 --with-coverage --cover-tests --cover-package=rdflib | ||
| - coverage run -m pytest |
There was a problem hiding this comment.
have a look at pytest-cov
It gives a report directly in the logs but also can produce additional reports (xml for coveralls and html for other use cases)
|
To clarify the scope here, I presume the goal is to get rid of everything nose related? |
|
I looked a bit at the test generation from Would like to get some feedback here before moving forward with that though. |
|
Just for the record below i pyyest log on top of 6.0.0. |
|
Would skipping these tests if the module isn't importable work?
https://docs.pytest.org/en/latest/how-to/skipping.html#how-to-use-skip-and-xfail-to-deal-with-tests-that-cannot-succeed
```
You can mark test functions that cannot be run on certain platforms or that
you expect to fail so pytest can deal with them accordingly and present a
summary of the test session, while keeping the test suite green.
A skip means that you expect your test to pass only if some conditions are
met, otherwise pytest should skip running the test altogether. Common
examples are skipping windows-only tests on non-windows platforms, or
skipping tests that depend on an external resource which is not available
at the moment (for example a database).
An xfail means that you expect a test to fail for some reason. A common
example is a test for a feature not yet implemented, or a bug not yet
fixed. When a test passes despite being expected to fail (marked with
pytest.mark.xfail), it’s an xpass and will be reported in the test summary.
pytest counts and lists skip and xfail tests separately. Detailed
information about skipped/xfailed tests is not shown by default to avoid
cluttering the output. You can use the -r option to see details
corresponding to the “short” letters shown in the test progress:
```
```bash
pytest -rxXs # show extra info on xfailed, xpassed, and skipped tests
```
https://docs.pytest.org/en/latest/how-to/skipping.html#skip-all-test-functions-of-a-class-or-module
```python
@pytest.mark.skipif(sys.platform == "win32", reason="does not run on
windows")
class TestPosixCalls:
def test_function(self):
"will not be setup or run under 'win32' platform"
```
```python
# test_module.py
pytestmark = pytest.mark.skipif(...)
```
…On Tue, Jul 20, 2021, 20:18 Tomasz Kłoczko ***@***.***> wrote:
Just for the record below i pyyest log on top of 6.0.0.
pytest is failing in units which requires berkeleydb module.
Currently most of the Linux distributions are trying move away from BDB
and use instead for example sqlite.
There are some number of pytest warnings.
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rdflib-6.0.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rdflib-6.0.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -k ' not test_service and not test_service_with_bind and not test_service_with_values'
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, httpbin-1.0.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, case-1.5.3, isort-1.3.0, aspectlib-1.5.2, asyncio-0.15.1, toolbox-0.5, xprocess-0.17.1, aiohttp-0.3.0, checkdocs-2.7.0, mock-3.6.1, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, flaky-3.7.0, hypothesis-6.14.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, Faker-8.10.0
collected 592 items / 7 deselected / 585 selected
. . [ 0%]
test/test_aggregate_graphs.py . [ 0%]
test/test_auditable.py ........................ [ 4%]
test/test_b64_binary.py . [ 4%]
test/test_batch_add.py .......... [ 6%]
test/test_canonicalization.py ...x [ 7%]
test/test_comparison.py .......... [ 8%]
test/test_conjunctive_graph.py ..x [ 9%]
test/test_container.py ................. [ 12%]
test/test_conventions.py . [ 12%]
test/test_create_input_source.py .. [ 12%]
test/test_csv2rdf.py .. [ 13%]
test/test_dataset.py sss... [ 14%]
test/test_datetime.py ....... [ 15%]
test/test_dawg.py x [ 15%]
test/test_diff.py ..x [ 15%]
test/test_duration.py ...... [ 16%]
test/test_empty_xml_base.py .. [ 17%]
test/test_evaluate_bind.py x [ 17%]
test/test_events.py . [ 17%]
test/test_expressions.py ...... [ 18%]
test/test_extras_external_graph_libs.py ss [ 19%]
test/test_finalnewline.py . [ 19%]
test/test_graph.py sssssssss......... [ 22%]
test/test_graph_context.py ssssssssss.......... [ 25%]
test/test_graph_formula.py x [ 25%]
test/test_graph_http.py .. [ 26%]
test/test_graph_items.py . [ 26%]
test/test_graph_operator.py ... [ 26%]
test/test_hex_binary.py .. [ 27%]
test/test_initbindings.py .......................... [ 31%]
test/test_issue084.py ........ [ 33%]
test/test_issue1043.py . [ 33%]
test/test_issue1141.py ... [ 33%]
test/test_issue1160.py . [ 33%]
test/test_issue160.py . [ 34%]
test/test_issue161.py . [ 34%]
test/test_issue184.py . [ 34%]
test/test_issue190.py ss [ 34%]
test/test_issue200.py . [ 34%]
test/test_issue209.py . [ 35%]
test/test_issue223.py . [ 35%]
test/test_issue247.py .. [ 35%]
test/test_issue248.py . [ 35%]
test/test_issue274.py ................................. [ 41%]
test/test_issue363.py .. [ 41%]
test/test_issue379.py .. [ 42%]
test/test_issue381.py ....... [ 43%]
test/test_issue432.py . [ 43%]
test/test_issue446.py . [ 43%]
test/test_issue492.py . [ 43%]
test/test_issue523.py . [ 44%]
test/test_issue532.py . [ 44%]
test/test_issue535.py . [ 44%]
test/test_issue545.py . [ 44%]
test/test_issue554.py . [ 44%]
test/test_issue563.py .. [ 45%]
test/test_issue579.py . [ 45%]
test/test_issue604.py . [ 45%]
test/test_issue655.py . [ 45%]
test/test_issue715.py . [ 45%]
test/test_issue733.py .. [ 46%]
test/test_issue801.py . [ 46%]
test/test_issue910.py .... [ 46%]
test/test_issue920.py . [ 47%]
test/test_issue923.py . [ 47%]
test/test_issue953.py . [ 47%]
test/test_issue977.py .. [ 47%]
test/test_issue_git_200.py . [ 47%]
test/test_issue_git_336.py . [ 48%]
test/test_literal.py ..................x [ 51%]
test/test_memory_store.py .. [ 51%]
test/test_mulpath_n3.py . [ 51%]
test/test_n3.py ................ [ 54%]
test/test_n3_suite.py x [ 54%]
test/test_namespace.py ........x.......... [ 58%]
test/test_nodepickler.py ... [ 58%]
test/test_normalizedString.py .... [ 59%]
test/test_nquads.py ........... [ 61%]
test/test_nquads_w3c.py x [ 61%]
test/test_nt_misc.py ...................... [ 65%]
test/test_nt_w3c.py x [ 65%]
test/test_parse_file_guess_format.py ... [ 65%]
test/test_parser.py .... [ 66%]
test/test_parser_helpers.py . [ 66%]
test/test_parser_reads_from_pathlike_object.py . [ 66%]
test/test_path_div_future.py . [ 66%]
test/test_prefixTypes.py . [ 67%]
test/test_preflabel.py ..... [ 67%]
test/test_prettyxml.py ....... [ 69%]
test/test_rdf_lists.py .. [ 69%]
test/test_rdfxml.py .. [ 69%]
test/test_roundtrip.py xx [ 70%]
test/test_seq.py .. [ 70%]
test/test_serialize.py .. [ 70%]
test/test_serializexml.py ..... [ 71%]
test/test_slice.py . [ 71%]
test/test_sparql.py ......... [ 73%]
test/test_sparql_agg_distinct.py .... [ 74%]
test/test_sparql_agg_undef.py x. [ 74%]
test/test_sparql_construct_bindings.py . [ 74%]
test/test_sparql_datetime.py .... [ 75%]
test/test_sparql_operators.py ... [ 75%]
test/test_sparql_parser.py .. [ 76%]
test/test_sparqlstore.py ........ [ 77%]
test/test_sparqlupdatestore.py ssssssssssss [ 79%]
test/test_sparqlupdatestore_mock.py . [ 79%]
test/test_store_berkeleydb.py FFFFFF [ 80%]
test/test_swap_n3.py x [ 80%]
test/test_term.py ....... [ 82%]
test/test_tokendatatype.py ..... [ 83%]
test/test_trig.py .........s. [ 84%]
test/test_trig_w3c.py x [ 85%]
test/test_trix_parse.py ... [ 85%]
test/test_trix_serialize.py .. [ 85%]
test/test_tsvresults.py . [ 86%]
test/test_turtle_serialize.py ..... [ 86%]
test/test_turtle_sort_issue613.py .. [ 87%]
test/test_turtle_w3c.py x [ 87%]
test/test_util.py ............................x............... [ 95%]
test/test_wide_python.py . [ 95%]
test/test_xmlliterals.py .... [ 95%]
test/jsonld/test_api.py . [ 96%]
test/jsonld/test_compaction.py x [ 96%]
test/jsonld/test_context.py ................ [ 98%]
test/jsonld/test_localsuite.py x [ 99%]
test/jsonld/test_named_graphs.py ... [ 99%]
test/jsonld/test_onedotone.py x [ 99%]
test/jsonld/test_testsuite.py x [100%]
================================================================================= FAILURES =================================================================================
____________________________________________________________________ BerkeleyDBTestCase.test_multigraph ____________________________________________________________________
self = <test.test_store_berkeleydb.BerkeleyDBTestCase testMethod=test_multigraph>
def setUp(self):
self.store_name = "BerkeleyDB"
self.path = mktemp()
> self.g = ConjunctiveGraph(store=self.store_name)
test/test_store_berkeleydb.py:11:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rdflib/graph.py:1500: in __init__
super(ConjunctiveGraph, self).__init__(store, identifier=identifier)
rdflib/graph.py:303: in __init__
self.__store = store = plugin.get(store, Store)()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <rdflib.plugins.stores.berkeleydb.BerkeleyDB object at 0x7fb666c25370>, configuration = None, identifier = None
def __init__(self, configuration=None, identifier=None):
if not has_bsddb:
> raise ImportError("Unable to import berkeleydb, store is unusable.")
E ImportError: Unable to import berkeleydb, store is unusable.
rdflib/plugins/stores/berkeleydb.py:69: ImportError
____________________________________________________________________ BerkeleyDBTestCase.test_open_shut _____________________________________________________________________
self = <test.test_store_berkeleydb.BerkeleyDBTestCase testMethod=test_open_shut>
def setUp(self):
self.store_name = "BerkeleyDB"
self.path = mktemp()
> self.g = ConjunctiveGraph(store=self.store_name)
test/test_store_berkeleydb.py:11:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rdflib/graph.py:1500: in __init__
super(ConjunctiveGraph, self).__init__(store, identifier=identifier)
rdflib/graph.py:303: in __init__
self.__store = store = plugin.get(store, Store)()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <rdflib.plugins.stores.berkeleydb.BerkeleyDB object at 0x7fb666db41c0>, configuration = None, identifier = None
def __init__(self, configuration=None, identifier=None):
if not has_bsddb:
> raise ImportError("Unable to import berkeleydb, store is unusable.")
E ImportError: Unable to import berkeleydb, store is unusable.
rdflib/plugins/stores/berkeleydb.py:69: ImportError
_______________________________________________________________________ BerkeleyDBTestCase.test_read _______________________________________________________________________
self = <test.test_store_berkeleydb.BerkeleyDBTestCase testMethod=test_read>
def setUp(self):
self.store_name = "BerkeleyDB"
self.path = mktemp()
> self.g = ConjunctiveGraph(store=self.store_name)
test/test_store_berkeleydb.py:11:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rdflib/graph.py:1500: in __init__
super(ConjunctiveGraph, self).__init__(store, identifier=identifier)
rdflib/graph.py:303: in __init__
self.__store = store = plugin.get(store, Store)()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <rdflib.plugins.stores.berkeleydb.BerkeleyDB object at 0x7fb666b5a850>, configuration = None, identifier = None
def __init__(self, configuration=None, identifier=None):
if not has_bsddb:
> raise ImportError("Unable to import berkeleydb, store is unusable.")
E ImportError: Unable to import berkeleydb, store is unusable.
rdflib/plugins/stores/berkeleydb.py:69: ImportError
__________________________________________________________________ BerkeleyDBTestCase.test_sparql_insert ___________________________________________________________________
self = <test.test_store_berkeleydb.BerkeleyDBTestCase testMethod=test_sparql_insert>
def setUp(self):
self.store_name = "BerkeleyDB"
self.path = mktemp()
> self.g = ConjunctiveGraph(store=self.store_name)
test/test_store_berkeleydb.py:11:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rdflib/graph.py:1500: in __init__
super(ConjunctiveGraph, self).__init__(store, identifier=identifier)
rdflib/graph.py:303: in __init__
self.__store = store = plugin.get(store, Store)()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <rdflib.plugins.stores.berkeleydb.BerkeleyDB object at 0x7fb67f72fcd0>, configuration = None, identifier = None
def __init__(self, configuration=None, identifier=None):
if not has_bsddb:
> raise ImportError("Unable to import berkeleydb, store is unusable.")
E ImportError: Unable to import berkeleydb, store is unusable.
rdflib/plugins/stores/berkeleydb.py:69: ImportError
___________________________________________________________________ BerkeleyDBTestCase.test_sparql_query ___________________________________________________________________
self = <test.test_store_berkeleydb.BerkeleyDBTestCase testMethod=test_sparql_query>
def setUp(self):
self.store_name = "BerkeleyDB"
self.path = mktemp()
> self.g = ConjunctiveGraph(store=self.store_name)
test/test_store_berkeleydb.py:11:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rdflib/graph.py:1500: in __init__
super(ConjunctiveGraph, self).__init__(store, identifier=identifier)
rdflib/graph.py:303: in __init__
self.__store = store = plugin.get(store, Store)()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <rdflib.plugins.stores.berkeleydb.BerkeleyDB object at 0x7fb67fa0baf0>, configuration = None, identifier = None
def __init__(self, configuration=None, identifier=None):
if not has_bsddb:
> raise ImportError("Unable to import berkeleydb, store is unusable.")
E ImportError: Unable to import berkeleydb, store is unusable.
rdflib/plugins/stores/berkeleydb.py:69: ImportError
______________________________________________________________________ BerkeleyDBTestCase.test_write _______________________________________________________________________
self = <test.test_store_berkeleydb.BerkeleyDBTestCase testMethod=test_write>
def setUp(self):
self.store_name = "BerkeleyDB"
self.path = mktemp()
> self.g = ConjunctiveGraph(store=self.store_name)
test/test_store_berkeleydb.py:11:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rdflib/graph.py:1500: in __init__
super(ConjunctiveGraph, self).__init__(store, identifier=identifier)
rdflib/graph.py:303: in __init__
self.__store = store = plugin.get(store, Store)()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <rdflib.plugins.stores.berkeleydb.BerkeleyDB object at 0x7fb67fc63df0>, configuration = None, identifier = None
def __init__(self, configuration=None, identifier=None):
if not has_bsddb:
> raise ImportError("Unable to import berkeleydb, store is unusable.")
E ImportError: Unable to import berkeleydb, store is unusable.
rdflib/plugins/stores/berkeleydb.py:69: ImportError
============================================================================= warnings summary =============================================================================
../../../../../usr/lib/python3.8/site-packages/_pytest/fixtures.py:223
/usr/lib/python3.8/site-packages/_pytest/fixtures.py:223: UserWarning: Code: _pytestfixturefunction is not defined in namespace XSD
fixturemarker: Optional[FixtureFunctionMarker] = getattr(
test/test_conjunctive_graph.py:49
test/test_conjunctive_graph.py:49: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_graph_ids will be ignored
def test_graph_ids():
test/test_dawg.py:516
test/test_dawg.py:516: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_dawg will be ignored
def test_dawg():
test/test_evaluate_bind.py:8
test/test_evaluate_bind.py:8: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_bind will be ignored
def test_bind():
test/test_graph_formula.py:129
test/test_graph_formula.py:129: PytestCollectionWarning: yield tests were removed in pytest 4.0 - testFormulaStores will be ignored
def testFormulaStores():
../../../../../usr/lib/python3.8/site-packages/_pytest/fixtures.py:223
/usr/lib/python3.8/site-packages/_pytest/fixtures.py:223: UserWarning: Code: _pytestfixturefunction is not defined in namespace DCAT
fixturemarker: Optional[FixtureFunctionMarker] = getattr(
../../../../../usr/lib/python3.8/site-packages/_pytest/fixtures.py:223
/usr/lib/python3.8/site-packages/_pytest/fixtures.py:223: UserWarning: Code: _pytestfixturefunction is not defined in namespace PROF
fixturemarker: Optional[FixtureFunctionMarker] = getattr(
../../../../../usr/lib/python3.8/site-packages/_pytest/fixtures.py:223
/usr/lib/python3.8/site-packages/_pytest/fixtures.py:223: UserWarning: Code: _pytestfixturefunction is not defined in namespace SDO
fixturemarker: Optional[FixtureFunctionMarker] = getattr(
../../../../../usr/lib/python3.8/site-packages/_pytest/fixtures.py:223
/usr/lib/python3.8/site-packages/_pytest/fixtures.py:223: UserWarning: Code: _pytestfixturefunction is not defined in namespace SOSA
fixturemarker: Optional[FixtureFunctionMarker] = getattr(
../../../../../usr/lib/python3.8/site-packages/_pytest/fixtures.py:223
/usr/lib/python3.8/site-packages/_pytest/fixtures.py:223: UserWarning: Code: _pytestfixturefunction is not defined in namespace SSN
fixturemarker: Optional[FixtureFunctionMarker] = getattr(
../../../../../usr/lib/python3.8/site-packages/_pytest/fixtures.py:223
/usr/lib/python3.8/site-packages/_pytest/fixtures.py:223: UserWarning: Code: _pytestfixturefunction is not defined in namespace TIME
fixturemarker: Optional[FixtureFunctionMarker] = getattr(
test/test_n3_suite.py:35
test/test_n3_suite.py:35: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_n3_writing will be ignored
def test_n3_writing():
test/test_nquads_w3c.py:27
test/test_nquads_w3c.py:27: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_nquads will be ignored
def test_nquads(tests=None):
test/test_nt_w3c.py:29
test/test_nt_w3c.py:29: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_nt will be ignored
def test_nt(tests=None):
test/test_roundtrip.py:90
test/test_roundtrip.py:90: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_cases will be ignored
def test_cases():
test/test_roundtrip.py:107
test/test_roundtrip.py:107: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_n3 will be ignored
def test_n3():
test/test_sparql_agg_undef.py:26
test/test_sparql_agg_undef.py:26: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_aggregates will be ignored
def test_aggregates():
test/test_swap_n3.py:98
test/test_swap_n3.py:98: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_cases will be ignored
def test_cases():
test/test_trig_w3c.py:70
test/test_trig_w3c.py:70: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_trig will be ignored
def test_trig(tests=None):
test/test_turtle_w3c.py:59
test/test_turtle_w3c.py:59: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_turtle will be ignored
def test_turtle(tests=None):
test/jsonld/test_compaction.py:255
test/jsonld/test_compaction.py:255: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_cases will be ignored
def test_cases():
test/jsonld/test_localsuite.py:27
test/jsonld/test_localsuite.py:27: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_suite will be ignored
def test_suite():
test/jsonld/test_onedotone.py:184
test/jsonld/test_onedotone.py:184: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_suite will be ignored
def test_suite():
test/jsonld/test_testsuite.py:77
test/jsonld/test_testsuite.py:77: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_suite will be ignored
def test_suite(skip_known_bugs=True):
test/translate_algebra/test_base.py:29
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/test/translate_algebra/test_base.py:29: PytestCollectionWarning: cannot collect test class 'Test' because it has a __init__ constructor (from: test/translate_algebra/test_base.py)
class Test:
test/translate_algebra/test_base.py:56
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/test/translate_algebra/test_base.py:56: PytestCollectionWarning: cannot collect test class 'TestExecution' because it has a __init__ constructor (from: test/translate_algebra/test_base.py)
class TestExecution:
test/test_finalnewline.py::testFinalNewline
test/test_nt_misc.py::NTTestCase::testIssue146
test/test_nt_misc.py::NTTestCase::testIssue78
test/test_sparql.py::test_sparql_update_with_bnode_serialize_parse
test/test_sparql_parser.py::SPARQLParserTests::test_insert_large
test/test_sparql_parser.py::SPARQLParserTests::test_insert_recursionlimit
test/test_xmlliterals.py::testRoundtrip
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/rdflib/plugins/serializers/nt.py:28: UserWarning: NTSerializer does not use custom encoding.
warnings.warn("NTSerializer does not use custom encoding.")
test/test_finalnewline.py::testFinalNewline
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/rdflib/plugins/serializers/jsonld.py:63: UserWarning: JSON should be encoded as unicode. Given encoding was: latin-1
warnings.warn(
test/test_finalnewline.py::testFinalNewline
test/test_nquads.py::NQuadsParserTest::test_serialize
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/rdflib/plugins/serializers/nquads.py:24: UserWarning: NQuadsSerializer does not use custom encoding.
warnings.warn("NQuadsSerializer does not use custom encoding.")
test/test_literal.py::TestParseBoolean::testNonFalseBoolean
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/rdflib/term.py:1430: UserWarning: Parsing weird boolean, 'abcd' does not map to True or False
warnings.warn(
test/test_literal.py::TestParseBoolean::testNonFalseBoolean
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/rdflib/term.py:1430: UserWarning: Parsing weird boolean, '10' does not map to True or False
warnings.warn(
test/test_namespace.py::NamespaceTest::test_iri
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/test/test_namespace.py:44: DeprecationWarning: Please use assertTrue instead.
self.assert_(ns["jörn"].startswith(ns))
test/test_namespace.py::NamespacePrefixTest::test_closed_namespace
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/test/test_namespace.py:213: UserWarning: DefinedNamespace does not address deprecated properties
warn("DefinedNamespace does not address deprecated properties")
test/test_preflabel.py::TestPrefLabel::test_default_label_sorting
test/test_preflabel.py::TestPrefLabel::test_default_preflabel_sorting
test/test_preflabel.py::TestPrefLabel::test_preflabel_lang_sorting_empty_lang_attr
test/test_preflabel.py::TestPrefLabel::test_preflabel_lang_sorting_en_lang_attr
test/test_preflabel.py::TestPrefLabel::test_preflabel_lang_sorting_no_lang_attr
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/rdflib/graph.py:779: DeprecationWarning: graph.preferredLabel() is deprecated and will be removed in rdflib 6.0.0.
warn(
test/test_prettyxml.py: 10 warnings
test/test_rdfxml.py: 2 warnings
test/test_serializexml.py: 10 warnings
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/rdflib/graph.py:1227: DeprecationWarning: graph.load() is deprecated, it will be removed in rdflib 6.0.0. Please use graph.parse() instead.
warn(
test/test_rdf_lists.py::OWLCollectionTest::testCollectionRDFXML
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/rdflib/plugins/serializers/rdfxml.py:262: UserWarning: Assertions on rdflib.term.BNode('Nc33941a1ba5b45e689864d24745e4a5e') other than RDF.first and RDF.rest are ignored ... including RDF.List
self.predicate(predicate, object, depth + 1)
test/test_seq.py::SeqTestCase::testSeq
/home/tkloczko/rpmbuild/BUILD/rdflib-6.0.0/rdflib/graph.py:929: DeprecationWarning: graph.seq() is deprecated and will be removed in rdflib 6.0.0.
warn(
test/test_util.py::TestUtilTermConvert::test_util_from_n3_expectliteralandlangdtype
/usr/lib64/python3.8/unittest/case.py:633: UserWarning: Code: fr is not defined in namespace XSD
method()
-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [1] test/test_dataset.py:131: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_dataset.py:78: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_dataset.py:157: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:355: couldn't find networkx
SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:355: couldn't find graph_tool
SKIPPED [1] test/test_graph.py:80: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph.py:138: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph.py:183: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph.py:215: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph.py:254: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph.py:83: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph.py:150: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph.py:329: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph.py:87: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph_context.py:110: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph_context.py:100: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph_context.py:170: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph_context.py:133: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph_context.py:117: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph_context.py:113: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph_context.py:195: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph_context.py:185: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph_context.py:146: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [1] test/test_graph_context.py:201: Dependencies for store 'BerkeleyDB' not available!
SKIPPED [2] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:355: Known issue with newlines in text
SKIPPED [12] ../../../../../usr/lib/python3.8/site-packages/_pytest/unittest.py:153: http://localhost:3031 is unavailable.
SKIPPED [1] test/test_trig.py:121: Iterative serialization currently produces 16 copies of everything
XFAIL test/test_canonicalization.py::TestConsistency::test_consistent_ids
reason:
XFAIL test/test_conjunctive_graph.py::test_graph_ids
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_graph_ids will be ignored
XFAIL test/test_dawg.py::test_dawg
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_dawg will be ignored
XFAIL test/test_diff.py::TestDiff::test_subsets
reason:
XFAIL test/test_evaluate_bind.py::test_bind
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_bind will be ignored
XFAIL test/test_graph_formula.py::testFormulaStores
reason: [NOTRUN] yield tests were removed in pytest 4.0 - testFormulaStores will be ignored
XFAIL test/test_literal.py::TestXsdLiterals::test_make_literals_ki
reason:
XFAIL test/test_n3_suite.py::test_n3_writing
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_n3_writing will be ignored
XFAIL test/test_namespace.py::ClosedNamespaceTest::test_repr_ef
reason:
XFAIL test/test_nquads_w3c.py::test_nquads
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_nquads will be ignored
XFAIL test/test_nt_w3c.py::test_nt
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_nt will be ignored
XFAIL test/test_roundtrip.py::test_cases
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_cases will be ignored
XFAIL test/test_roundtrip.py::test_n3
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_n3 will be ignored
XFAIL test/test_sparql_agg_undef.py::test_aggregates
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_aggregates will be ignored
XFAIL test/test_swap_n3.py::test_cases
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_cases will be ignored
XFAIL test/test_trig_w3c.py::test_trig
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_trig will be ignored
XFAIL test/test_turtle_w3c.py::test_turtle
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_turtle will be ignored
XFAIL test/test_util.py::TestUtilTermConvert::test_util_from_n3_not_escapes_xf
reason:
XFAIL test/jsonld/test_compaction.py::test_cases
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_cases will be ignored
XFAIL test/jsonld/test_localsuite.py::test_suite
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_suite will be ignored
XFAIL test/jsonld/test_onedotone.py::test_suite
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_suite will be ignored
XFAIL test/jsonld/test_testsuite.py::test_suite
reason: [NOTRUN] yield tests were removed in pytest 4.0 - test_suite will be ignored
FAILED test/test_store_berkeleydb.py::BerkeleyDBTestCase::test_multigraph - ImportError: Unable to import berkeleydb, store is unusable.
FAILED test/test_store_berkeleydb.py::BerkeleyDBTestCase::test_open_shut - ImportError: Unable to import berkeleydb, store is unusable.
FAILED test/test_store_berkeleydb.py::BerkeleyDBTestCase::test_read - ImportError: Unable to import berkeleydb, store is unusable.
FAILED test/test_store_berkeleydb.py::BerkeleyDBTestCase::test_sparql_insert - ImportError: Unable to import berkeleydb, store is unusable.
FAILED test/test_store_berkeleydb.py::BerkeleyDBTestCase::test_sparql_query - ImportError: Unable to import berkeleydb, store is unusable.
FAILED test/test_store_berkeleydb.py::BerkeleyDBTestCase::test_write - ImportError: Unable to import berkeleydb, store is unusable.
============================================ 6 failed, 517 passed, 39 skipped, 7 deselected, 22 xfailed, 70 warnings in 27.47s =============================================
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1268 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAMNS5E6W7BXWQS7MATD5DTYYG65ANCNFSM4YMVJ63A>
.
|
|
@kloczek so you say BerkeleyDB should be avoided for the future. I think this is related to #1347 |
|
@aucampia The goal is to move to a modern easy to maintain and easy to use test framework, so that developers use the tests and want to write tests for missing and newly introduced code. |
I think that can make sense. |
Yes, please do allow BerkeleyDB tests to be skipped if it's not installed. We (I) want to keep BerkeleyDB as a Store option as we currently don't have any other local file system DB options. Please do feel free to complete SQLite per perhaps the started Shelve Stores though, that would be great! |
cclauss
reviewed
Oct 12, 2021
| @@ -535,16 +535,13 @@ def test_dawg(): | |||
| setFlags() | |||
|
|
|||
| if SPARQL10Tests: | |||
| for t in nose_tests(testers, "test/DAWG/data-r2/manifest-evaluation.ttl"): | |||
| yield t | |||
| return [t for t in nose_tests(testers, "test/DAWG/data-r2/manifest-evaluation.ttl")] | |||
There was a problem hiding this comment.
Why these changes? It will take both time and memory to build up this list. The original could be rewritten as:
Suggested change
| return [t for t in nose_tests(testers, "test/DAWG/data-r2/manifest-evaluation.ttl")] | |
| yield from nose_tests(testers, "test/DAWG/data-r2/manifest-evaluation.ttl") |
There was a problem hiding this comment.
If this is a straightforward code improvement with a generator, great!
There was a problem hiding this comment.
Exceptions within list comprehensions are also sometimes less optimal for debuggability, so yield probably is preferable here (though pytest has great functionality for parametrization such that each would be loop iteration is a named test) but couldn't you just change the [ ] to ( ) to return an - is it a - generator comprehension instead of building a list without a way to drop into a debugger upon exception in a named test function?
There was a problem hiding this comment.
Different options for test IDs¶
pytest will build a string that is the test ID for each set of values in a parametrized test. These IDs can be used with -k to select specific cases to run, and they will also identify the specific case when one is failing. Running pytest with --collect-only will show the generated IDs.Numbers, strings, booleans and None will have their usual string representation used in the test ID. For other objects, pytest will make a string based on the argument name
https://docs.pytest.org/en/latest/example/parametrize.html#different-options-for-test-ids
cclauss
reviewed
Oct 14, 2021
| - python setup.py install | ||
| - flake8 --exit-zero rdflib | ||
| - PYTHONWARNINGS=default nosetests --with-timer --timer-top-n 42 --with-coverage --cover-tests --cover-package=rdflib | ||
| - coverage run -m pytest | ||
| - coverage report |
There was a problem hiding this comment.
Perhaps add the Python 3.10 tests in #1439 and then that failing PR can be closed.
|
@ashleysommer @white-gecko @edmondchuc @aucampia @westurner @cclauss @FlorianLudwig @rchateauneu It seems our biggest rdflib issues now are to do with our testing framework, automated testing and automated style/language and other tests to enhance consistency. The small mess here is preventing us from moving to support Python 3.10 and a number of other things. If you are keen to assist with rdflib development in the short term, please could you all prioritize this PR to remove nose, implement pytest and then configure Drone to work with pytest for all Python versions? If we can get this done, we will likely then be able to develop new features much more easily. Thanks. |
|
@nicholascar I will focus my efforts here until it is done. I am not sure what the best way is to proceed, I think one option is to make PRs against @white-gecko 's branch, otherwise I can make a new branch continuing from this one and add additional changes needed. It would be nice if we could find some way of doing this incrementally but I am unsure what options there are for that and will look into it. |
|
@aucampia perhaps if we move this branch to an rdflib branch, i.e. not a branch on @white-gecko's fork, then it will be easier to raise PR against that? @white-gecko, can you please move this to an rdflib branch? Also, you might like to rebase now that #1267's merged, as you indicated in the description. |
|
The drop in coverage seems to be at least partly related to manifest testing. With pytest basically none of those were running, and I remember now why I made this comment about subtests earlier on. I changed manifest testing to use pytest_subtest now and this seems to work. I think there are other options with pytest that pytest-subtest, but they require restructuring tests significantly as far as I know. I will try make it work at least with subtest and then we can consider what to do from there on. There are also still a couple of places where nose is referenced directly. Coverage after that change: https://coveralls.io/jobs/88589568 |
|
OK, I'm happy to pick up and review a whole pile of test in detail too. I'll work out which set - parsers or serialisers or stores etc. and ensure I know how many tests there actually are there, what proportion are operating and what the nose/pytest differences are. |
|
I have thought more about manifest tests, and there is another way to do them than with pytest-subtests, and I will make make some as an example. However, I can't really think of a very clean way to integrate either way with what is happening inside Lines 535 to 637 in d9e86c9 I think the best approach is to just scrap that (and other similar code) for now. I'm not sure if something is relying on it. I think it would be nice if there was a test manifest executor that can be run outside of pytest, which is essentially what is happening there as far as I can tell, but maybe this can be done in a way that is more generic and can be reused in other places. Not sure if we should do that now however, possibly it's best to just get rid of that for now. |
|
Actually, I guess we do want to be able to write earl reports still. So i will look at running tests manifests outside of pytest to generate this, but maybe not with highest priority. But any ideas here will be welcome. |
|
I see there is |
|
Most of the earl test reports haven’t been regenerated for a very, very, long time! It would be good to regenerate them. Even if the Working Groups (RDF etc) that they were originally for aren’t around any more. We might consider putting the results up somewhere in a form other than RDF to show off RDFLib’s results against those tests. |
|
I don’t think anything is relying on the test files’ Yes, convert I guess part of all of this is to find out what all these testutils.py and manifest.py are all doing… |
|
Earl_report would probably be generally useful as a pytest plugin? JUnit XML output support is in core: https://docs.pytest.org/en/latest/how-to/writing_plugins.html https://github.com/pytest-dev/pytest-reportlog entry_points={"pytest11": ["pytest_reportlog = pytest_reportlog.plugin"]},https://github.com/pytest-dev/pytest-reportlog/blob/master/src/pytest_reportlog/plugin.py https://github.com/LaserPhaser/pytest-collect-formatter https://docs.pytest.org/en/latest/how-to/writing_plugins.html |
|
thanks for the notes @westurner - will have a look at pytest plugins, it may make the most sense |
|
Hitting some failures with Update 1:Seems to be somewhat state dependent, if I just run the two failing test I don't get problems. |
|
Does setting PYTHONHASHSEED result in test output variance?
https://docs.python.org/3/using/cmdline.html#cmdoption-R
…On Tue, Oct 19, 2021, 17:16 Iwan Aucamp ***@***.***> wrote:
Hitting some failures with rdflib.compare.isomorphic on drone which I
can't reproduce locally.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1268 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAMNSYVZJZV5AWU453TCBDUHXN2LANCNFSM4YMVJ63A>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
|
Actually I think it is related ALLOW_LISTS_OF_LISTS here: rdflib/test/jsonld/test_testsuite.py Lines 99 to 107 in 1468000 But I'm still figuring it out. |
|
Indeed, if test_suide_dir is module scoped, should it be setting a module
global in order to chdir
…On Tue, Oct 19, 2021, 17:45 Iwan Aucamp ***@***.***> wrote:
Actually I think it is related to this:
https://github.com/RDFLib/rdflib/blob/1468000d8db3021718fb7bbbdc7249ecee8f3b7a/test/jsonld/test_testsuite.py#L99-L107
But I'm still figuring it out.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1268 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAMNS6EY3NYQM7XBDSHOK3UHXRI3ANCNFSM4YMVJ63A>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
|
Okay all good, there were a couple of places where global state was not cleaned up after modifying it, but now it seems fine. Coverage is now up to 78.3% https://coveralls.io/builds/43624040 - still 4 percentage points lower than master. |
|
I found some more " |
|
Actually I think the remaining coverage is doctests, will see what to do about that. |
|
Was doctest run on rst files also? I tried now but it seems to not work well: Can of course be fixed, but just wondering if it was working before. If not we can defer doing it till later. Update 1:Command was: .venv/bin/pytest -x --doctest-modules --doctest-glob="*.rst" docs/ |
|
Okay coverage is back to where it was before (even slightly higher):
I suspect it is because some things were ignored with nosetest that are not ignored with pytest: nosetest ignored: Line 10 in d9e86c9 pytest ignores: https://github.com/iafork/rdflib/blob/ad795edb92a84c8b40a8ab23469d38517c8f2a6b/setup.cfg#L34-L37 |
|
I think mostly what is left now is to fix up reporting. |
|
Did json earl reporting work on master? I tried running it but it does not work for me: I suspect I just don't know how to run it. I would like to reproduce it as closely as possible. |
I think it's not been run for a very long time so the answer is effectively no but, as you say, a probable yes if looked in to. |
|
I will try finish the reporting and publish #1452 this weekend |
|
I now have EARL reports working, details can be found in https://github.com/iafork/rdflib/blob/6f6ffa199ee0ebadd1225fefbe524707c1ddc0ac/test/README.rst. It is done with a pytest plugin, and when the plugin is enabled it will create an EARL assertion for every test with a There is still some final touches needed.
|
|
There is also some warning relating to the EARL reporting plugin which I can't quite make sense of: |
|
Example reports can be found here: https://filebin.net/w8kplw7c0tdr6c07 |
|
"JSON-LD 1.1 Processor Conformance
https://w3c.github.io/json-ld-streaming/reports/
|
|
@westurner thanks, I did compare the pytest EALR outputs to the reports listed for w3c: For jsonld 1.0:
For jsonld 1.1
One obvious difference is that we are running significantly fewer tests, but it's the same for master and the pytest branch: This should probably be looked into, but can be done later. |
|
#1452 is ready for review, will continue discussion there. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This needs to be rebased after #1267 is merged.
Proposed Changes
The test coverage is not the same, some tests are not yet executed.
Check: https://docs.pytest.org/en/stable/nose.html
yieldtests are not supported by pytest. (This was how they worked in nose) Some are changed to direct calls, but maybe we want to actually have test generators in these cases. We might use https://hypothesis.readthedocs.io/en/latest/quickstart.html for this.The migration is no simple task. Additional contributions are very welcome!