JSON repr not always working

[liamhuber]

The HasToDict mix-in class used throughout the workflow module adds a custom _repr_json_ method to get nice notebook representations for the various objects. Unfortunately, this isn't working for a couple of objects (originally discussed in PR #658).

M(ish)WE:

from pyiron_contrib.workflow import Workflow

wf = Workflow("with_prebuilt")

wf.structure = wf.add.atomistics.BulkStructure(repeat=3, cubic=True, element="Al")
wf.engine = wf.add.atomistics.Lammps(structure=wf.structure)
wf.calc = wf.add.atomistics.CalcMd(job=wf.engine)
wf.calc.signals.input.run = wf.engine.signals.output.ran

• wf is not good (Workflow(Node))
• wf.structure is fine (SingleValue(Function(Node)))
• wf.structure.inputs is no good (Inputs)
• wf.structure.inputs.cubic is fine (DataChannel)

As a temporary fix in the linked PR, I added a repr_json() method that just explicitly wrapped the returned dictionary in _repr_json_ in a IPython.display.JSON call before returning it, and this always works fine -- so it's just when the notebook tries to do it's magic that we run into trouble.

I think I see the issue already (it has to do with overloading __getattr__), but I want the solution process documented. I hope to raise and close this in a sitting.

[liamhuber]

Asking the notebook to show me the docstring gives a good hint where things are going wrong. I.e. wf.structure._repr_json_? gives me something boring and expected, but wf._repr_json_ raises

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[10], line 1
----> 1 get_ipython().run_line_magic('pinfo', 'wf._repr_json_')

File ~/anaconda3/envs/pyiron_311/lib/python3.11/site-packages/IPython/core/interactiveshell.py:2417, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth)
   2415     kwargs['local_ns'] = self.get_local_scope(stack_depth)
   2416 with self.builtin_trap:
-> 2417     result = fn(*args, **kwargs)
   2419 # The code below prevents the output from being displayed
   2420 # when using magics with decodator @output_can_be_silenced
   2421 # when the last Python token in the expression is a ';'.
   2422 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):

File ~/anaconda3/envs/pyiron_311/lib/python3.11/site-packages/IPython/core/magics/namespace.py:58, in NamespaceMagics.pinfo(self, parameter_s, namespaces)
     56     self.psearch(oname)
     57 else:
---> 58     self.shell._inspect('pinfo', oname, detail_level=detail_level,
     59                         namespaces=namespaces)

File ~/anaconda3/envs/pyiron_311/lib/python3.11/site-packages/IPython/core/interactiveshell.py:1798, in InteractiveShell._inspect(self, meth, oname, namespaces, **kw)
   1796     pmethod(info.obj, oname, formatter)
   1797 elif meth == 'pinfo':
-> 1798     pmethod(
   1799         info.obj,
   1800         oname,
   1801         formatter,
   1802         info,
   1803         enable_html_pager=self.enable_html_pager,
   1804         **kw,
   1805     )
   1806 else:
   1807     pmethod(info.obj, oname)

File ~/anaconda3/envs/pyiron_311/lib/python3.11/site-packages/IPython/core/oinspect.py:799, in Inspector.pinfo(self, obj, oname, formatter, info, detail_level, enable_html_pager, omit_sections)
    775 """Show detailed information about an object.
    776 
    777 Optional arguments:
   (...)
    796 - omit_sections: set of section keys and titles to omit
    797 """
    798 assert info is not None
--> 799 info_b: Bundle = self._get_info(
    800     obj, oname, formatter, info, detail_level, omit_sections=omit_sections
    801 )
    802 if not enable_html_pager:
    803     del info_b["text/html"]

File ~/anaconda3/envs/pyiron_311/lib/python3.11/site-packages/IPython/core/oinspect.py:755, in Inspector._get_info(self, obj, oname, formatter, info, detail_level, omit_sections)
    729 def _get_info(
    730     self,
    731     obj: Any,
   (...)
    736     omit_sections=(),
    737 ) -> Bundle:
    738     """Retrieve an info dict and format it.
    739 
    740     Parameters
   (...)
    752         Titles or keys to omit from output (can be set, tuple, etc., anything supporting `in`)
    753     """
--> 755     info_dict = self.info(obj, oname=oname, info=info, detail_level=detail_level)
    756     bundle = self._make_info_unformatted(
    757         obj,
    758         info_dict,
   (...)
    761         omit_sections=omit_sections,
    762     )
    763     return self.format_mime(bundle)

File ~/anaconda3/envs/pyiron_311/lib/python3.11/site-packages/IPython/core/oinspect.py:855, in Inspector.info(self, obj, oname, info, detail_level)
    853 parents_docs = None
    854 prelude = ""
--> 855 if info and info.parent is not None and hasattr(info.parent, HOOK_NAME):
    856     parents_docs_dict = getattr(info.parent, HOOK_NAME)
    857     parents_docs = parents_docs_dict.get(att_name, None)

File ~/work/pyiron/pyiron_contrib/pyiron_contrib/workflow/composite.py:241, in Composite.__getattr__(self, key)
    240 def __getattr__(self, key):
--> 241     return self.nodes[key]

KeyError: '__custom_documentations__'

[liamhuber]

Ok, super. The issue was that in IPython/core/oinspect.py:855 above, info.parent is our Workflow (well, and Composite(Node) in this case) or IO object that overrides __getattr__ to try and access the nodes or channels dict, respectively. But this I do with key access, so when it's not there __getattr__ raises a KeyError instead of an AttributeError, which wrecks the hasattr call.

I'm making a PR now to give these classes' __getattr__ more informative and correctly typed exceptions.