return a value for compare_asdf

[braingram]

There are many uses of compare_asdf in the regression tests like the following:

romancal/romancal/regtest/test_wfi_pipeline.py

Line 43 in 68e6e0d

assert compare_asdf(rtdata.output, rtdata.truth, **ignore_asdf_paths) is None

These asserts are always True as compare_asdf does not return a value (so it's output will always be None).

romancal/romancal/regtest/regtestdata.py

Lines 528 to 532 in 68e6e0d

	def compare_asdf(result, truth, **kwargs):
	f = StringIO()
	asdf_diff([result, truth], minimal=False, iostream=f, **kwargs)
	if f.getvalue():
	f.getvalue()

This PR updates compare_asdf to output the string output of the diff or None if the string is empty (as is seen when the compared files are identical).

Regression tests running: https://plwishmaster.stsci.edu:8081/blue/organizations/jenkins/RT%2FRoman-Developers-Pull-Requests/detail/Roman-Developers-Pull-Requests/351/pipeline/205

Checklist

• added entry in CHANGES.rst under the corresponding subsection
• updated relevant tests
• updated relevant documentation
• updated relevant milestone(s)
• added relevant label(s)

[braingram]

Given the nature of the change it seems unlikely that the regression tests will pass cleanly. So far I'm seeing 14 errors (only 10 are expected due to missing webbpsf data, see #865).

[codecov]

Codecov Report

Patch coverage is 50.00% of modified lines.

Files Changed	Coverage
romancal/regtest/regtestdata.py	50.00%

📢 Thoughts on this report? Let us know!.

[braingram]

The output is less-than-ideal.

For the failing test_linearity_step the output is:

AssertionError: assert '\x1b[32m> tree:\x1b[0m\n\x1b[32m> roman:\x1b[0m\n\x1b[32m>   border_ref_pix_left:\x1b[0m\n\x1b[32m>     value:\x1b[0m...1b[0m\n        refpix:\x1b[0m\n\x1b[32m>         \x1b[0m\n\x1b[31m<         crds://roman_wfi_refpix_0045.asdf\x1b[0m\n' is None

The special characters don't appear to be formatted anywhere in the test output (so simply returning the diff result is likely not the preferred solution). Even passing it through a print yields a less-than-helpful result (the color is lost in the copy-and-paste into this github comment):

> tree:
> roman:
>   border_ref_pix_left:
>     value:...1b[0m
        refpix:
>
<         crds://roman_wfi_refpix_0045.asdf

I am also suspicious that the current ignore_asdf_paths fixture is not up-to-date:

romancal/romancal/regtest/conftest.py

Lines 203 to 212 in 68e6e0d

	@pytest.fixture
	def ignore_asdf_paths():
	ignore_attr = [
	"meta.[date, filename]",
	"asdf_library",
	"history",
	"cal_logs",
	]
	return {"ignore": ignore_attr}

based on the output of the failing test_level2_grism_processing_pipeline test:

tree:
  roman:
    cal_logs:
      -
>       2023-09-13T20:30:00.768Z :: stpi...5418
        ndarrays differ by contents
        ndarrays differ by contents

[braingram]

The devdeps failure looks unrelated to the changes in this PR and appear to be an issue with webbpsf and numpy 2.0. I opened an issue as it does not appear that webbpsf yet tests against numpy 2.0 and might be effected by the numpy scalar repr changes:
spacetelescope/webbpsf#737

[schlafly]

Thanks Brett. Yeah, we're running on parallel paths. FWIW, I updated ignore attrs as follows as I attempt to figure out what else is broken:

diff --git a/romancal/regtest/conftest.py b/romancal/regtest/conftest.py
index 3ea3681..7664d43 100644
--- a/romancal/regtest/conftest.py
+++ b/romancal/regtest/conftest.py
@@ -203,10 +203,10 @@ def rtdata_module(artifactory_repos, envopt, request, jail):
 @pytest.fixture
 def ignore_asdf_paths():
     ignore_attr = [
-        "meta.[date, filename]",
+        "roman.meta.[date, filename]",
         "asdf_library",
         "history",
-        "cal_logs",
+        "roman.cal_logs",
     ]
 
     return {"ignore": ignore_attr}

[schlafly]

Re when we should merge this PR---I'd like to scope out a little better what fraction of the issues this PR identifies are related to limitations of asdf.commands.diff and what fraction are errors we need to address on the romancal side before we merge.

[braingram]

Re when we should merge this PR---I'd like to scope out a little better what fraction of the issues this PR identifies are related to limitations of asdf.commands.diff and what fraction are errors we need to address on the romancal side before we merge.

Sounds great. I'll make this draft until we can sort out the errors.

[braingram]

Closing in favor of #868