Backport of new pileup jet ID for MiniAOD re-processing

[ahinzmann]

Backport of the following PRs:
#10398 jbrands:newOfflinePuID_new
#10730 ahinzmann:fixpuid
#10625 ahinzmann:PUjetIDcleanup76
#10923 ahinzmann:pileupjetiddqm76

They contain a new training of the pileup jet ID with 74 MC.
This is not the final/optimal training, but already better than the 53 training on 74 MC (ROC curves are in #10625). Optimization of a working point for MVAmet is also not yet done, thus it is not configured to be used in MVAmet.

runTheMatrix -l limited works in CMSSW_7_5_X_2015-08-27-2300

@gpetruc this can be included in the MiniAOD processing

[cmsbuild]

A new Pull Request was created by @ahinzmann for CMSSW_7_5_X.

Backport of new pileup jet ID for MiniAOD re-processing

It involves the following packages:

DQM/PhysicsHWW
DQMOffline/JetMET
DataFormats/JetReco
PhysicsTools/PatAlgos
RecoJets/JetProducers
RecoMET/METPUSubtraction

@cvuosalo, @monttj, @cmsbuild, @deguio, @slava77, @vadler, @danduggan can you please review it and eventually sign? Thanks.
@rappoccio, @imarches, @yslai, @acaudron, @mmarionncern, @jdolen, @nhanvtran, @schoef, @ferencek, @gpetruc, @mariadalfonso, @pvmulder, @TaiSakuma, @rociovilar this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@Degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

[slava77]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.

[gpetruc]

Thanks, I have added it to the table in https://twiki.cern.ch/twiki/bin/view/CMS/MiniAODSpring15pass2 and I'll add the 74X one as soon as I get it.

[cmsbuild]

+1
Tested at: f42c90c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11002/7717/summary.html

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11002/7717/summary.html

[slava77]

+1

for #11002 f42c90c

• backport of the listed PRs from 76X is done correctly
• jenkins tests pass and comparisons with baseline show expected behavior:
  • monitored reco products are unchanged
  • DQM plots show differences in cleanedJets distributions (includes ID) and in PU ID distributions of uncleaned jets
• tested locally in CMSSW_7_5_X_2015-08-27-2300 /test area sign583/ on run2 2015B data in 251721 DoubleEG 200 events promptReco with " @standardDQM+@miniAODDQM+@common" DQM pieces included
  • monitored reco products are unchanged; DQM plots show changes in cleaned jet distributions and in PU jet ID variables for uncleaned jets
  • memory RSS peak goes down by 58 MB
  • allocated MEM_LIVE after 10 events goes down by 26 MB (from TMVA::Reader::BookMVA)
  • RECO and AOD outputs are unchanged. MINIAOD has tiny variation in size of patJets_slimmedJets, which should be from changes in stored values (PU IDs in this case)
  • timing of the modules reduces

   -1.726185      -0.62%        77.63 ms/ev ->         5.70 ms/ev pileupJetIdEvaluatorDQM
   -0.756911      -0.11%        22.48 ms/ev ->        10.13 ms/ev hwwAnalyzer
   -0.537214      -0.02%         6.72 ms/ev ->         3.87 ms/ev pileupJetIdEvaluatorCHSDQM
   -0.536477      -0.03%         9.38 ms/ev ->         5.41 ms/ev pileupJetId

[slava77]

@monttj @deguio @danduggan please check this PR
Thank you.

[deguio]

+1

[monttj]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_5_X IBs once checked with relvals in the development release cycle of CMSSW (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @smuzaffar

[slava77]

@davidlange6 please check this one.
Both 74X and 76X versions are in the release already

[davidlange6]

+1