Reading histograms with attached fit functions

[FlorianBen]

Dear experts,

Newcomer’s question. I’m using more and more uproot for histogram post-processing but, sometime, I have some issue to get histograms from root files. I put a MWE here and a root file as sample:

#!/usr/bin/env python
import uproot

def main():
    file = uproot.open('scripts/output.root')
    print(file.classnames()) # Show 2 TH1D in file
    h1neutron_amp = file['h1neutron_amp;1'] # Get first hist
    print(h1neutron_amp.values()) # Work fine
    h1neutron_rate = file['h1neutron_rate;1'] # Traceback here
    print(h1neutron_rate.values())

if __name__ == '__main__':
    main()

The file contains two TH1D, I can get the ‘h1neutron_amp’ without problem. But when I try to get the other one, I get this traceback:

Traceback (most recent call last):
  File "/local/home/fb250757/Documents/scripts/uproot_hist.py", line 15, in <module>
    main()
  File "/local/home/fb250757/Documents/scripts/uproot_hist.py", line 10, in main
    h1neutron_rate = file['h1neutron_rate;1']
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/reading.py", line 2106, in __getitem__
    return self.key(where).get()
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/reading.py", line 2509, in get
    out = cls.read(chunk, cursor, context, self._file, selffile, parent)
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/model.py", line 1296, in read
    versioned_cls.read(
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/model.py", line 806, in read
    self.read_members(chunk, cursor, context, file)
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/models/TH.py", line 1586, in read_members
    file.class_named("TH1", 8).read(
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/model.py", line 806, in read
    self.read_members(chunk, cursor, context, file)
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/models/TH.py", line 538, in read_members
    self._members["fFunctions"] = file.class_named("TList").read(
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/model.py", line 806, in read
    self.read_members(chunk, cursor, context, file)
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/models/TList.py", line 50, in read_members
    item = uproot.deserialization.read_object_any(
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/deserialization.py", line 271, in read_object_any
    obj = cls.read(chunk, cursor, context, file, selffile, parent)
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/model.py", line 1296, in read
    versioned_cls.read(
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/model.py", line 806, in read
    self.read_members(chunk, cursor, context, file)
  File "<dynamic>", line 17, in read_members
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/deserialization.py", line 271, in read_object_any
    obj = cls.read(chunk, cursor, context, file, selffile, parent)
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/model.py", line 1296, in read
    versioned_cls.read(
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/model.py", line 806, in read
    self.read_members(chunk, cursor, context, file)
  File "<dynamic>", line 10, in read_members
  File "/local/home/fb250757/Documents/Prog/Python/Env/Science/lib/python3.8/site-packages/uproot/containers.py", line 1249, in read
    raise NotImplementedError(
NotImplementedError: non-memberwise serialization of AsMap
in file scripts/output.root

Unfortunately, the traceback is a too cryptic for my understanding. Do you have an idea on where this error may come from, please?

Thanks for your help.

OS: Ubuntu 20.04
python: 3.8.10
uproot: 4.3.5

[jpivarski]

This is an unimplemented part of ROOT serialization. (That's why the error type is NotImplementedError.) There's an open issue in Uproot to deal with it, and an example file containing this type has been identified: #556. So it's just a matter of somebody getting to it and implementing it. Since you ran into it, I'll bump it up in my mental priority queue.

[Moelf]

btw how old is this file, it seems to be made with a relatively old ROOT version

[FlorianBen]

Hello, the file in the attachment is just a small part of a bigger ROOT file generated by compiled routines. I recompiled the routines recently with a modern setup (ROOT 6.26/02).

However, I didn't dive too much in the code base of the routines, so maybe they rely on deprecated functions or other old ROOT magic.

[jpivarski]

My other example of a file with non-memberwise std::map serialization (#556) is also a histogram, and the std::map is the parameterization of a TFormula, which is a fit function (TF1) attached to the histogram. Your stack trace also shows that it failed while deserializing fFunctions, so I'm going to guess that it's the same thing. I've released a fix in Uproot 4.3.6. Try upgrading and seeing if it fixes the problem.

(In retrospect, I should have asked you to test on the git branch before merging it, because we could have adapted to any surprises that you encounter. But, oh well, at least it will be easier for you to test. Just pip install -U uproot and try again. If it doesn't work, then I'll have to do another patch-release.)

[FlorianBen]

Hello,
Indeed, the issue was only for TH1s with fitting functions attached. I upgraded uproot to 4.3.6 with pip, and uproot opens correctly the histograms now (I tested several, not all of them).
Thanks for your swift actions.