MET Filter update for mAOD reprocessing

[schoef]

• The HBHE Iso Noise filter flag is promoted to a separate boolean in mAOD.
• The eeBadSC filter is added to the recommended MET filters flag.
• Thresholds for the ecalDeadCellFilter and the ecalBoundaryEnergyFilter are updated. The latter flag is added to mAOD
• Fixing the Flag_goodVertices in mAOD
• Moving the scraping and the primary vertex filter into the RecoMet/MetFilters

This is an update of #10406

[cmsbuild]

A new Pull Request was created by @schoef for CMSSW_7_4_X.

MET Filter update for mAOD reprocessing

It involves the following packages:

CalibMuon/DTCalibration
PhysicsTools/PatAlgos
RecoMET/METFilters

@diguida, @cerminar, @monttj, @cmsbuild, @vadler, @mmusich can you please review it and eventually sign? Thanks.
@rappoccio, @imarches, @ahinzmann, @acaudron, @mmarionncern, @jdolen, @nhanvtran, @ferencek, @gpetruc, @mariadalfonso, @pvmulder, @TaiSakuma this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@Degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

[schoef]

@Slava
I tested according to your proposal in #10406.
I got some errors from the pluginlibraryloader in a wider test but this shouldn't be a problem of this code.
I'm done with commits, please test.

[mmusich]

please test

[cmsbuild]

The tests are being triggered in jenkins.

[cmsbuild]

-1
Tested at: 79e14c5
When I ran the RelVals I found an error in the following worklfows:
4.22 step2

runTheMatrix-results/4.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log

5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

4.53 step2

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODreHLT+HARVESTDreHLT/step2_RunPhoton2012B+RunPhoton2012B+HLTD+RECODreHLT+HARVESTDreHLT.log

8.0 step1

runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step1_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log

9.0 step1

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step1_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

25.0 step1

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaEle+HARVEST+ALCATT/step1_TTbar+TTbar+DIGI+RECOAlCaEle+HARVEST+ALCATT.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

1306.0 step1

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step1_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

1330.0 step1

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step1_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log

101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log

25202.0 step1

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step1_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

50202.0 step1

runTheMatrix-results/50202.0_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50/step1_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50.log

140.53 step2

runTheMatrix-results/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/step2_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

1003.0 step2

runTheMatrix-results/1003.0_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM/step2_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM.log

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-10604/193831/summary.html

[slava77]

@schoef
Robert, you could have just updated the old PR (well, if it was easy ;) )
The errors seem legitimate:

e.g. in 4.53

from PhysicsTools.PatAlgos.slimming.metFilterPaths_cff   import *
  File "/tmp/cmsbuild/workspace/ib-any-integration/CMSSW_7_4_X_2015-08-05-2300/python/PhysicsTools/PatAlgos/slimming/metFilterPaths_cff.py", line 5, in <module>
    from RecoMET.METFilters.metFilters_cff import HBHENoiseFilterResultProducer, HBHENoiseFilter, HBHENoiseIsoFilter, CSCTightHaloFilter, hcalLaserEventFilter, EcalDeadCellTriggerPrimitiveFilter, eeBadScFilter, ecalLaserCorrFilter, EcalDeadCellBoundaryEnergyFilter, primaryVertexFilter 
ImportError: cannot import name HBHENoiseIsoFilter

it looks like you missed some commits

[cmsbuild]

+1
Tested at: 120e850
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-10604/7025/summary.html

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-10604/7025/summary.html

[slava77]

+1

for #10604 120e850

• code changes are as expected
  • note that the good PV filter previously defined in ALCARECODtCalibHI_cff.py with a 15 cm cut is now defined in primaryVertexFilter_cfi.py with a 24 cm range in z
  • among the running modules (using prompt reco setup for run2): trackingFailureFilter and goodVertices are removed; HBHENoiseIsoFilter and EcalDeadCellBoundaryEnergyFilter are added
• same code as in 75X and in 76X version
• jenkins tests passed and comparisons with the baseline show no relevant differences
• tested additionally with run2 prompt reco setup in CMSSW_7_4_X_2015-08-09-1100 /test area test10604/: using doubleMuon PD file from run 251721:
  • there is no significant change in CPU: the new EcalDeadCellBoundaryEnergyFilter takes ~ 5 ms/evt (HBHENoiseIsoFilter is almost invisible)
  • MEM_LIVE based on igprof is essentially unchanged: I observed a 4.5 MB increase after 10 events, but wasn't able to trace it to a modified module. Due to the change in introduced modules the call tree is slightly different and the change in memory appeared to point to untouched modules (like tracking or pixel clustering) suggesting some memory paging fluctuations
  • the list persisted products in RECO/AOD/MINIAOD is unchanged

[monttj]

+1

[slava77]

@mmusich @diguida please check, AlCa signature is needed

[mmusich]

+1
clean up of ALCARECODtCalibHI and ALCARECODtCalib configurations.

[mmusich]

@slava77 sorry for the delay

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_4_X IBs once checked with relvals in the development release cycle of CMSSW (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @smuzaffar

[davidlange6]

closing PRs included in #10649