NanoAOD prototype [RFC] #20563
NanoAOD prototype [RFC] #20563
Conversation
…e eta's differ, and point is not the associated PV
…from track (which may not be available)
…tag v1.0 except data files
…truc/NanoAOD.git tag v1.0 (to be later rebased out of the history)
|
The code-checks are being triggered in jenkins. |
|
+code-checks |
|
A new Pull Request was created by @gpetruc (Giovanni Petrucciani) for master. It involves the following packages: DataFormats/PatCandidates The following packages do not have a category, yet: PhysicsTools/NanoAOD @perrotta, @cmsbuild, @monttj, @slava77 can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
why are they even needed?
|
|
they're needed for nano-AOD production (in full CMSSW) since some high-level ids or calibrations are derived after miniAOD production (or, they can be reapplied on miniAOD and we don't always re-miniAOD when they change). |
|
btw, reading nano still needs calibration input such as JECs.
slava would you join tomorrow cross pog to discuss the integration of this?
the idea is to have a roundtable discussion on the NanoAods
Il 18 set 2017 9:41 PM, "Giovanni Petrucciani" <[email protected]>
ha scritto:
… they're needed for nano-AOD production (in full CMSSW) since some
high-level ids or calibrations are derived after miniAOD production (or,
they can be reapplied on miniAOD and we don't always re-miniAOD when they
change).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#20563 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AEyiliym1z_RiQzsaXiVGRQ6tp2PU6U7ks5sjsbOgaJpZM4PbFog>
.
|
|
hi all - i started the process of getting the data files into a separate repo - but one comment - perhaps its a good moment to unify the xml dat txt (maybe yaml?) variants of text file endings/formats to store what must be the same sort of information ? |
|
@davidlange6 the xmls are TMVA stuff, so not all the same. |
Add Jet recalibration
gzip xml for MVA
|
The code-checks are being triggered in jenkins. |
|
+code-checks |
| // | ||
|
|
||
| template <typename T> | ||
| class BaseMVAValueMapProducer : public edm::stream::EDProducer<> { |
There was a problem hiding this comment.
It would be useful to use a GlobalCache here, no? (to reduce total number of weight-file reads in multithreaded mode)
Similar to: https://github.com/cms-sw/cmssw/blob/master/RecoEgamma/EgammaTools/interface/MVAValueMapProducer.h
cms-sw/cmsdist#3431 is merged already (23 hrs ago) |
|
how about the DF? do you want us to create DataFormats/NanoAOD ?
…On Thu, Sep 21, 2017 at 6:06 PM, Slava Krutelyov ***@***.***> wrote:
Data files (e.g. xml trainings of MVAs) are all in one commit, and will be
rebased out of the history once we get a new cms-data package for
PhysicsTools/NanoAOD
cms-sw/cmsdist#3431 <cms-sw/cmsdist#3431> is
merged already (23 hrs ago)
time to rebase?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#20563 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AEyilo12SWw2Gkj-rDSDbS_ySVs9waMIks5skomNgaJpZM4PbFog>
.
|
|
On 9/21/17 9:10 AM, arizzi wrote:
how about the DF? do you want us to create DataFormats/NanoAOD ?
commonly persisted classes should be in the DataFormats subsystem
|
|
are you ok with a new DF directory or would you prefer to include it in
existing ones (in case, which one?)
On Thu, Sep 21, 2017 at 6:15 PM, Slava Krutelyov <[email protected]>
wrote:
… On 9/21/17 9:10 AM, arizzi wrote:
> how about the DF? do you want us to create DataFormats/NanoAOD ?
commonly persisted classes should be in the DataFormats subsystem
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#20563 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AEyilsvx44BcAtp4p-xlyu1HUDc0G3FGks5skou6gaJpZM4PbFog>
.
|
|
On 9/21/17 9:20 AM, arizzi wrote:
are you ok with a new DF directory or would you prefer to include it in
existing ones (in case, which one?)
separate is perhaps a better choice.
It will be lightweight and with minimal dependences
|
|
closing while I do the rebase and cleanup to avoid spamming everyone with each change to the branch. will re-open when done |
|
Relocated dataformats and rebased removing the data files from the history. To run locally in 9_4_0_pre1 now one needs |
|
On 9/22/17 4:45 AM, Giovanni Petrucciani wrote:
Relocated dataformats and rebased removing the data files from the
history. To run locally in 9_4_0_pre1 now one needs
|git clone https://github.com/cms-data/PhysicsTools-NanoAOD.git
PhysicsTools/NanoAOD/data|
I merged also a few other pending developments in the content, and the
integration with cmsDriver (d7dc162
<d7dc162>)
for development purposes it is cleaner to clone into the "external" area:
mkdir -p $CMSSW_BASE/external/$SCRAM_ARCH/data
git clone https://github.com/cms-data/PhysicsTools-NanoAOD.git
$CMSSW_BASE/external/$SCRAM_ARCH/data/PhysicsTools/NanoAOD/data
this way there is no interference with the src files coming from
cms-sw/cmssw repo
…
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#20563 (comment)>, or
mute the thread
<https://github.com/notifications/unsubscribe-auth/AEdcboA5EGhP10BYt_qxBuUN5EouZr9Pks5sk53OgaJpZM4PbFog>.
|
|
cloning in the external area doesn't work very nicely with crab... |
First PR of NanoAOD prototype, mostly to collect comments on the technical implementation of it.
Based on CMSSW_9_4_0_pre1 + #20491, importing all code from https://github.com/gpetruc/NanoAOD except the python post-processing, which we want to put in a standalone package
Data files (e.g. xml trainings of MVAs) are all in one commit, and will be rebased out of the history once we get a new cms-data package for
PhysicsTools/NanoAODThis package define three persistent dataformats (FlatTable, MergeableCounterTable, UniqueString) that are however only needed when NanoAOD is written in full EDM Format.
They could be relocated to a
DataFormats/NanoAODor similar if you think it would be useful. The dataformats are content-agnostic, so we don't expect them to evolve much in the future.Some other generic classes may be relocated to
PhysicsTools/PatAlgos.Integration with cmsDriver & runTheMatrix is not yet included in this PR, but we probably want to define:
NANO?)NANOAOD,NANOAODSIM?); we may have to define it for both the full EDM and the nano format, which have the same event content but differ in the output module used (PoolOuputModulevsNanoAODOutputModule), advice on how to do this is welcome.@arizzi @peruzzim @emanueledimarco