Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

podio::Frame based I/O #94

Closed
tmadlener opened this issue Feb 14, 2023 · 6 comments
Closed

podio::Frame based I/O #94

tmadlener opened this issue Feb 14, 2023 · 6 comments
Assignees
Labels
enhancement New feature or request

Comments

@tmadlener
Copy link
Contributor

Since the currently used EventStore based I/O functionality is de-facto deprecated and will be removed from podio, we should switch to the Frame based I/O model. Ideally we only use podio components here and do not roll a separate implementation of Readers/Writers. Another goal is to support parallelism of any kind (inter-event and intra-event) out of the box, so that we can also easily switch all algorithm developments to the Gaudi Functional approach. To make this work we have to fulfill the correct Gaudi interface. We (@vvolkl and me) had a brief meeting with Marco Clemencic from the Gaudi team trying to identify the correct one. I will try to put the summary of that meeting here, although it will potentially not bring us all the way to the end, and we might have to have another meeting. (Obviously, the possibility exists that I misunderstood a few things)

  • Gaudi Functional uses the underlying EvtSvc to populate its inputs/outputs (it is after all only an Algorithm)
  • For multithreading Gaudi will use several transient stores in parallel (these are managed by the HiveWhiteBoard(?)), which effectively hides the switching between the different stores via an EventContext
  • The AlgTask (GaudiHive) is managing this context switching and also calls sysExecute on the managed algorithms to actually run them
  • The IDataProviderSvc seems to be the interface that we need to fulfill. The basic idea would be to make each instance of this service own exactly one podio::Frame and leave the rest of the management to Gaudi
@tmadlener tmadlener added the enhancement New feature or request label Feb 14, 2023
@hegner
Copy link
Contributor

hegner commented Apr 20, 2023

@tmadlener - at the moment I am stuck at https://github.com/key4hep/k4FWCore/blob/master/k4FWCore/include/k4FWCore/DataHandle.h#L82
with the special use cases not being properly handled in the Frame.
Do you know how widespread that use actually is?

@tmadlener
Copy link
Contributor Author

I think as far as Key4hep is concerned, this is not a use case that we support any longer. This should be handled by the podio::UserDataCollection. Similar for the floating_point cases below. Having said that, I don't know if it is used "in the wild", but if it is used, switching to the UserDataCollection would in any case be the way forward, I think.

@hegner
Copy link
Contributor

hegner commented Apr 20, 2023

great. then I will just kill it for good

@hegner
Copy link
Contributor

hegner commented Apr 21, 2023

Seems we actually have our very basic tests all depend on it :-(
So we need to find out whether it is really used in production

@hegner
Copy link
Contributor

hegner commented Apr 21, 2023

Discussed in issue #69

@tmadlener
Copy link
Contributor Author

Done with #100

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants