Replacing SpikeNoise flag by NegativeNoise one

[abdoulline]

Followup on the presentation (by THONG Nguyen) and a subsequent discussion at PPD General https://indico.cern.ch/event/452395/
:
Replacement of HBHSpikeNoise (not quite efficient in 25ns data) with "universal" (good both for 25ns and 50ns) HBHNegativeNoise rechit filter.

[cmsbuild]

A new Pull Request was created by @abdoulline (Salavat Abdullin) for CMSSW_7_6_X.

Replacing SpikeNoise flag by NegativeNoise one

It involves the following packages:

Configuration/StandardSequences

@cmsbuild, @franzoni, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @makortel, @GiacomoSguazzoni, @rovere, @VinInn, @Martin-Grunewald, @cerati, @dgulhan this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@Degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

[slava77]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.

[cmsbuild]

-1
Tested at: aa7aec4
The relvals timed out after 2 hours.

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11667/8673/summary.html

[abdoulline]

There is some ongoing (HCAL) discussion whether NEF should be "generalized" (retroactively applied to Run 1) as in this PR,
or it would better replace SpikeNoise only for Run 2 (in customiseDataRun2Common)...

[slava77]

On 10/7/15 4:06 PM, Salavat Abdullin wrote:

There is some ongoing (HCAL) discussion whether NEF should be
"generalized" (retroactively applied to Run 1) as in this PR,
or it better replace SpikeNoise only for Run 2 (in
customiseDataRun2Common)...

It better be general for all, if it applies to both 50ns and 25 ns.
Why not?

If it applies to 50 ns in run2, why would it not for run1 inputs?
The HBHE hardware relevant to the physics performance is the same, isn't it?

—
Reply to this email directly or view it on GitHub
#11667 (comment).

[abdoulline]

Yes, I agree (that's why I've submitted this PR) we should just simply 

replace SpikeNoise with NegativeNoise everywhere (25/50ns, Run1/2).

BTW, test has failed due to reason(s) unrelated to PR itself, I believe...
Salavat

On Wed, 7 Oct 2015, Slava Krutelyov wrote:

On 10/7/15 4:06 PM, Salavat Abdullin wrote:

There is some ongoing (HCAL) discussion whether NEF should be
"generalized" (retroactively applied to Run 1) as in this PR,
or it better replace SpikeNoise only for Run 2 (in
customiseDataRun2Common)...

It better be general for all, if it applies to both 50ns and 25 ns.
Why not?

If it applies to 50 ns in run2, why would it not for run1 inputs?
The HBHE hardware relevant to the physics performance is the same, isn't it?

—
Reply to this email directly or view it on GitHub
#11667 (comment).

—
Reply to this email directly or view it on
GitHub.[AEx02hgiFB_N0wto_JHi2PTrOAhB9r0wks5o5YGzgaJpZM4GKmMo.gif]

[slava77]

@cmsbuild please test

timeout is not an answer

[cmsbuild]

The tests are being triggered in jenkins.

[cmsbuild]

-1

Tested at: aa7aec4
I found errors in the following addon tests:

cmsDriver.py RelVal -s HLT:50nsGRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_50nsGRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --customise=SLHCUpgradeSimulations/Configuration/postLS1Customs.customisePostLS1_50ns --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_50nsGRun_DATA.root --fileout file:RelVal_Raw_50nsGRun_DATA_HLT_RECO.root : FAILED - time: date Thu Oct 8 00:41:47 2015-date Thu Oct 8 00:37:16 2015 s - exit: 16640
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --customise=SLHCUpgradeSimulations/Configuration/postLS1Customs.customisePostLS1 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Thu Oct 8 00:50:53 2015-date Thu Oct 8 00:47:53 2015 s - exit: 16640
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --customise=SLHCUpgradeSimulations/Configuration/postLS1Customs.customisePostLS1 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Thu Oct 8 00:57:46 2015-date Thu Oct 8 00:52:53 2015 s - exit: 16640

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11667/8678/summary.html

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11667/8678/summary.html

[slava77]

Here are some notes
for #11667 aa7aec4

• code /config change is as expected
• small single-hit level changes are expected and observed only in data processing in both jenkins and in local tests
• tested locally in CMSSW_7_6_0_pre6 (before the "TFormula" and other problems crept in to the IBs) with the run2 matrix (134.{7,8}0{2,3,5} workflows including DoubleEG, DoubleMuon, and Met PDs) as well as prompt reco setup in DoubleEG and DoubleMu in 251721 and 256729)
  • there is a small change also in run1 workflows 2011A and 2011B minbias (1000.0 and 4.29)
  • MET PD shows differences; nothing in other run2 workflows. These look in line with noise cleanup in HBHE (red is new), the following plots are from the 134.805 (MET PD). Here we have =more= hits passing
    
    
    with all jets considered, changes are not that noticeable
    
    after 50 GeV threshold selection the picture is more clear, but is less expected
    
    The increase in neutral hadron fraction here is also visible in MET
    

I know it's just a few events and there is some selection bias (MET PD), but purely neutral hadron 50 GeV or above jets and an increase in MET NHF is not something really expected.

[abdoulline]

Well, we know NEF is not exactly SpikeNoise. As to the (acceptable) level of changes, I leave it for comments of Halil -
@dertexaner

[davidlange6]

@abdoulline @dertexaner - does this need more review from hcal or not? Please clarify.

[dertexaner]

Hi all;
This PR is fine from the HCAL noise side.
Halil

[slava77]

I was running on more events from 134.805 to see something more convincing

Salavat and Halil, were the tests presented in PPD yesterday done on top of the HBHESpikeNoise or really in the way this PR is doing?

[dertexaner]

Hi Slava;
The tests were replacing HBHESpikeNoise with NEF.
Halil

[slava77]

ah, OK then. The PR is consistent with the plots used in the PPD.

[slava77]

So, with 10K events processed, it's clear that this PR generates more MET and also clearly more high-pt PF jets with only neutral hadron component

these jets are bumping up around 100 GeV in pt

MET increases, generally

In 9.5K events out of 10K pfMET doesn't change, if I look for events where it actually changes, the result is pretty large increase

And some plot from the standard DQM:

So, MET increases, high-pt jets with neutral-hadron-only component increases.
Why can this be considered an improvement?
Maybe the dataset is bad somehow.

@abdoulline @dertexaner please comment

[dertexaner]

As a quick comment, here is our findings with HBHE calo MET in 2015C NoBPTX data (though the comparison here is not spike filter vs NEF cleaned MET):

[slava77]

Would this mean that the kind of noise in NoBPTX is different from what's in the (inclusive) MET PD?

[dertexaner]

The relative composition of different noise sources is probably different, but we should be mostly sensitive to the spike-like component (though the old spike-like filter and NEF don't exactly have the same scope). Some neutral component and MET increase could be due to the fact that spike-filter turn on (vs pulse charge in TS4+TS5+TS6) is a bit faster for the SpikeNoise flag than the NegativeNoise, and the peak efficiency is also ~10% (relative) different.

[slava77]

@schoef @mariadalfonso

[slava77]

Running 10K events on 134.802 (DoubleEG dataset) shows essentially no change (one or two events with small change). So, this makes sense.

[slava77]

@schoef @mariadalfonso
could you please push for some early checks of the effect of this PR on the MET side.
My observations suggest that things get worse, but that's a limited test without much of an authoritative value.
Thank you.

[dertexaner]

As an additional suggestion, if possible, it might be worthwhile to have these validation plots with the baseline HBHE run2 filter (HBHENoiseFilterResultRun2Loose) applied since neither Spike nor NEF will be applied alone in a MET dataset.