Nested TDirectories

[jpivarski]

The TDirectory code is very likely naive— hard-coding assumptions that only hold true for the root directory. It should be possible to generalize this and therefore add nested directories.

This is probably related to question #128, in which @sbinet created such a file in Go-HEP, but had trouble reading it in ROOT. There seems to be something subtle about TDirectories.

The Python interface should still be through __setitem__. In this case, a name with slashes (/) should be used to navigate through nested directories, just as it is for reading data. (This is why there's a stub of "if /, raise NotImplementedError.") Don't create subdirectory objects (as is possible in the reading code) because the distributed mutability would be a nightmare.

[reikdas]

Don't create subdirectory objects (as is possible in the reading code) because the distributed mutability would be a nightmare.

If we are not going to add support for writing objects in the subdirectories, are the nested TDirectories just going to be empty TDirectories? (or with just more empty TDirectories inside them)

[jpivarski]

What I was thinking when I wrote that was that creating objects in subdirectories would have the same interface as creating objects in the base directory:

file["obj1"] = something
file["dir1/obj2"] = something_else

and the directory would be created or increased on the fly as needed to construct that name. In other words, there would never be any empty directories (like git or S3, in which directories don't really exist—they're just names with slashes).

The complexity this saves is that the user never gets access to any directory object they can modify at will—adding new objects (and possibly new TDirectories) always goes through the central file object, which you can control to expand with new blocks as necessary.

This kind of thing doesn't need block management—we can afford to reserve, say, 10 slots in a new directory and then hide it as junk bytes and create a new one with 100 slots (etc.) when it fills up.

[mneira10]

Hi! Is how is this feature going? It would be incredibly useful!