Gating ctest on Fedora latest is failing on a ssg python module test

[ggbecker]

It appears the test started failing by itself, no changes made in the code could indicate something like this. I'll continue investigating. It fails only on latest Fedora container. Other distros seem to be ok.

This could be related #10411 but the gating for that pull request worked just fine in this same gating test.

For example, from: https://github.com/ComplianceAsCode/content/actions/runs/4600335639/jobs/8126802809

we have

___________________________ test_rule_to_xml_element ___________________________

rule_accounts_tmout = <ssg.build_yaml.Rule object at 0x7fbb87617290>

    def test_rule_to_xml_element(rule_accounts_tmout):
        xmldiff_main = pytest.importorskip("xmldiff.main")
        rule_el = rule_accounts_tmout.to_xml_element()
        with temporary_filename() as real:
            ET.ElementTree(rule_el).write(real)
            expected = os.path.join(DATADIR, "accounts_tmout.xml")
>           diff = xmldiff_main.diff_files(real, expected)

../../tests/unit/ssg-module/test_build_yaml.py:446: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.11/site-packages/xmldiff/main.py:51: in diff_files
    return _diff(
/usr/local/lib/python3.11/site-packages/xmldiff/main.py:37: in _diff
    return diff_trees(
/usr/local/lib/python3.11/site-packages/xmldiff/main.py:27: in diff_trees
    return list(diffs)
/usr/local/lib/python3.11/site-packages/xmldiff/diff.py:435: in diff
    etree.register_namespace(k, v)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   ValueError: Prefix format reserved for internal use

[ggbecker]

I've found a difference in the pip package xmldiff:
passes on 2.5

 Collecting xmldiff
  Downloading xmldiff-2.5-py3-none-any.whl (59 kB)

Fails on 2.6

 Collecting xmldiff
  Downloading xmldiff-2.6-py3-none-any.whl (42 kB)

We can mitigate the problem by installing the older version instead xmldiff==2.5 so we don't block PRs and then we can find a permanent solution.

[ggbecker]

The change in 2.6 has affected our project.

https://github.com/Shoobx/xmldiff/blob/0caf0f136982fa66498e9c77c97faf517ef4a544/CHANGES.rst

Shoobx/xmldiff@422528b

@jan-cerny maybe you will understand better what's going on with the tests. But I suspect it's the namespace thing, the namespace of an sample xml and what is loaded by build_yaml.py:Rule.to_xml_element might be different and is causing the issue when doing the diff.

To reproduce the issue you just need to install xmldiff v2.6.

pip install xmldiff==2.6

[jan-cerny]

@ggbecker

Great investigation! Yes, it's indeed the namespaces!

the namespace of an sample xml and what is loaded by build_yaml.py:Rule.to_xml_element might be different and is causing the issue when doing the diff.

They aren't different. I have checked it, I have extracted the files that are compared and I have seen both "real" and "expected' file have the same and correct namespace.

I have found that the ValueError Exception doesn't come directly from xmldiff but instead from register_namespace from xml.etree.ElementTree which is now called in xmldiff. This functions is used to register namespace prefixes, but, it can't be used to register the prefixes starting with ns, eg. ns0. These prefixes are used by xml.etree.ElementTree as default prefixes if the program that creates the XML document doesn't call register_namespace, so logically you can't register these.

So the problem is specifically in the way how namespace prefixes are named.

I was able to sucessfully fix/workaround the traceback posted on the top of the issue by this:

1. in the test code, registering all the namespace prefixes that we use, we have a handy function register_namespaces in ssg/xml.py to do that
2. renaming the ns.* prefixes in the test data to a registered prefix, ie. replace by sed all occurrences of ns0 to xccdf-1.2 in tests/unit/ssg-module/data/accounts_tmout.xml

I guess that the other tracebacks in the test job output can be fixed in a similar way.

However, I'm not sure if we should do this fix or if this is a bug in the xmldiff, I think that the xmldiff should support diffing files that are created by xml.etree.elementree without explicit namespace registration used during the creation. So maybe we should report a bug to them?

[ggbecker]

@ggbecker

Great investigation! Yes, it's indeed the namespaces!

the namespace of an sample xml and what is loaded by build_yaml.py:Rule.to_xml_element might be different and is causing the issue when doing the diff.

They aren't different. I have checked it, I have extracted the files that are compared and I have seen both "real" and "expected' file have the same and correct namespace.

I have found that the ValueError Exception doesn't come directly from xmldiff but instead from register_namespace from xml.etree.ElementTree which is now called in xmldiff. This functions is used to register namespace prefixes, but, it can't be used to register the prefixes starting with ns, eg. ns0. These prefixes are used by xml.etree.ElementTree as default prefixes if the program that creates the XML document doesn't call register_namespace, so logically you can't register these.

So the problem is specifically in the way how namespace prefixes are named.

I was able to sucessfully fix/workaround the traceback posted on the top of the issue by this:

1. in the test code, registering all the namespace prefixes that we use, we have a handy function `register_namespaces` in `ssg/xml.py` to do that

2. renaming the `ns.*` prefixes in the test data to a registered prefix, ie. replace by `sed` all occurrences of `ns0` to `xccdf-1.2` in `tests/unit/ssg-module/data/accounts_tmout.xml`

I guess that the other tracebacks in the test job output can be fixed in a similar way.

However, I'm not sure if we should do this fix or if this is a bug in the xmldiff, I think that the xmldiff should support diffing files that are created by xml.etree.elementree without explicit namespace registration used during the creation. So maybe we should report a bug to them?

Yes, it sounds like we should report it to https://github.com/Shoobx/xmldiff/ at least to make them aware and potentially explain if we are wrong if that's the case.

[jan-cerny]

I have opened issue in xmldiff: Shoobx/xmldiff#108

[jan-cerny]

@ggbecker they are quick, they have released a new version 2.6.1 that probably fixes it: https://pypi.org/project/xmldiff/

[ggbecker]

We need to test with this new package and remove the workaround that installs the 2.5 package