add triplets using TOB and TIB4 for redundancy in TobTec and PIxelLess seeding steps respectively. reduce chi2-cut in MuonInOut #11492

VinInn · 2015-09-25T14:21:02Z

trying to mitigate inactive feds in TOB L3.
for an extensive description please refer to
https://indico.cern.ch/event/442135/session/1/contribution/18/attachments/1160196/1670021/tob3.pdf

for "ideal" MC results by @makortel
Plots are here:
https://mkortela.web.cern.ch/mkortela/tracking/validation/CMSSW_7_6_0_pre5_tob3/index.html

In the global picture there is ~no effect in efficiency, while fake rate increases by ~0.5-1 % (up to 3 % in barrel)
https://mkortela.web.cern.ch/mkortela/tracking/validation/CMSSW_7_6_0_pre5_tob3/ttbar_25ns_ootb/effandfake1.pdf

For tobTecStep itself also an efficiency increase is visible
https://mkortela.web.cern.ch/mkortela/tracking/validation/CMSSW_7_6_0_pre5_tob3/ttbar_25ns_tobTecStep/effandfake1.pdf
but it is so small (~0.05 %-unit) that it is not visible in the global picture.

some timing numbers:

            760pre5    +tob3    increase

reco 11.27 11.32 ~0.4 %
tracking 5.80 5.87 1 %

tobTecStep 0.426 0.463 8 %
*SeedsTripl 0.080 0.096 20 %
*TrackCandidates 0.256 0.276 8 %
*Tracks 0.028 0.028 ~0
(for other parts of tobTecStep the times are ~the same)

All times are real time per event as given by
process.options.wantSummary=True for a single run of 200 events of
TTbar+25ns PU on the same machine. For both cases the reported event
loop real time and CPU time are within <0.5 %, so I think the numbers
are good enough.

cmsbuild · 2015-09-25T14:21:12Z

A new Pull Request was created by @VinInn (Vincenzo Innocente) for CMSSW_7_6_X.

add triplets using TOB4 for redundancy

It involves the following packages:

RecoTracker/IterativeTracking

@cmsbuild, @cvuosalo, @slava77 can you please review it and eventually sign? Thanks.
@ghellwig, @makortel, @GiacomoSguazzoni, @rovere, @VinInn, @mschrode, @istaslis, @gpetruc, @cerati, @dgulhan this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@Degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

VinInn · 2015-09-25T14:22:07Z

@boudoul FYI

slava77 · 2015-09-25T14:44:44Z

@cmsbuild please test

cmsbuild · 2015-09-25T14:46:17Z

The tests are being triggered in jenkins.

cmsbuild · 2015-09-25T16:40:38Z

+1
Tested at: c84dcd0
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11492/8398/summary.html

cmsbuild · 2015-09-25T19:32:35Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11492/8398/summary.html

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

cmsbuild · 2015-09-29T08:11:20Z

Pull request #11492 was updated. @cvuosalo, @monttj, @cmsbuild, @deguio, @slava77, @ggovi, @vadler, @vanbesien, @danduggan can you please check and sign again.

VinInn · 2015-09-29T08:12:54Z

apparently there where some newer PR integrated after last IB... (rebase worked!)

cmsbuild · 2015-09-30T22:10:43Z

+1
Tested at: bb306e1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11492/8531/summary.html

cmsbuild · 2015-09-30T23:13:37Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11492/8531/summary.html

makortel · 2015-10-01T07:22:35Z

I've updated the plots in https://mkortela.web.cern.ch/mkortela/tracking/validation/CMSSW_7_6_0_pre5_pr11492/index.html to contain the results from bb306e1.

VinInn · 2015-10-01T07:44:38Z

Thanks Matti.
In my opinion the gain in efficiency justifies the increase of the fake rate in particular for the current PU.
We need MC with PU15 and realistic conditions to confirm.
On the other hand I do not think we can hold this PR for longer

VinInn · 2015-10-01T08:09:57Z

In general I think we have a problem with MixelTriplets (and piccolos/tobtec)
tracks with less than 5 hits are meaningless for these steps
https://mkortela.web.cern.ch/mkortela/tracking/validation/CMSSW_7_6_0_pre5_pr11492/ttbar_25ns_highPurity_mixedTripletStep/hitsAndPt.pdf
It means that at least one seed hit has been dropped (and not much added...)

slava77 · 2015-10-01T10:28:14Z

Can this all be done better with knowledge of bad components? Meaning, to
add looser seeds or go to another layer if there is a compatible dead or
bad on the way?
On Oct 1, 2015 3:09 AM, "Vincenzo Innocente" [email protected]
wrote:

In general I think we have a problem with MixelTriplets (and
piccolos/tobtec)
tracks with less than 5 hits are meaningless for these steps

https://mkortela.web.cern.ch/mkortela/tracking/validation/CMSSW_7_6_0_pre5_pr11492/ttbar_25ns_highPurity_mixedTripletStep/hitsAndPt.pdf
It means that at least one seed hit has been dropped (and not much
added...)

—
Reply to this email directly or view it on GitHub
#11492 (comment).

VinInn · 2015-10-01T11:39:40Z

On 1 Oct, 2015, at 12:28 PM, Slava Krutelyov [email protected] wrote:

Can this all be done better with knowledge of bad components? Meaning, to
add looser seeds or go to another layer if there is a compatible dead or
bad on the way?
not in the current seeding “framework”
v.

slava77 · 2015-10-01T12:40:40Z

I suppose no more changes to this PR are coming in a follow up to #11492 (comment)
@VinInn please confirm
Thank you.

VinInn · 2015-10-01T12:49:35Z

@slava77devel
no. This PR will not be modified further

slava77 · 2015-10-02T00:40:04Z

Here are some observations from me based on
CMSSW_7_6_X_2015-09-29-0900

1313:
almost everything is from 7-hit tracks

25202 has about the same pattern in differences

134.803 (2015C DoubleMuon)

It looks like out of the increase in the number of generalTracks, about 2/3 of them make it to pf charged hadrons, somewhat independent of the sample (3TeV dijet in 1313, ttbar with PU in 25202, pp run2 data in 134.803).

Looking per iteration (Matti's plots have better statistics), in 25202 and even in 1313, it is clear that tobTec fake rate blows up, practically all from a single bin in hit multiplicity:

pixelLess seems ~OK

tobTec not

The primary effect is not the addition of the triplet with MTOB4, rather the decreased requirement on the number of hits.

Looking at times:
2015C relval workflow 134.802

   +0.084656      +0.11%       102.39 ms/ev ->       111.44 ms/ev pixelLessStepSeeds
   +0.272941      +0.28%        70.43 ms/ev ->        92.70 ms/ev pixelLessStepTrackCandidates
   +0.172652      +0.08%        33.24 ms/ev ->        39.53 ms/ev pixelLessStepTracks

   +0.174884      +0.04%        15.94 ms/ev ->        18.99 ms/ev tobTecStepSeedsTripl
   +0.051277      +0.03%        49.22 ms/ev ->        51.81 ms/ev tobTecStepTrackCandidates
   +0.142365      +0.02%        11.48 ms/ev ->        13.24 ms/ev tobTecStepTracks

this is about 0.5%

25202:

   +0.078327      +0.06%       671.03 ms/ev ->       725.73 ms/ev pixelLessStepSeeds
   +0.378757      +0.21%       397.15 ms/ev ->       582.71 ms/ev pixelLessStepTrackCandidates
   +0.193496      +0.03%       134.65 ms/ev ->       163.49 ms/ev pixelLessStepTracks

   +0.195250      +0.03%       120.96 ms/ev ->       147.13 ms/ev tobTecStepSeedsTripl
   +0.086133      +0.03%       318.42 ms/ev ->       347.08 ms/ev tobTecStepTrackCandidates
   +0.319044      +0.02%        39.90 ms/ev ->        55.05 ms/ev tobTecStepTracks

this is about 2% increase

I suppose, the arguments are that this fake rate increase is not so large at lower pileup, which is more relevant to 2015 pp data taking and we will figure out what to do better for 2016. Right?

Still, does it make sense to make it all up out of the short tracks in tobtec?

VinInn · 2015-10-02T07:47:50Z

@slava77 I agree that for PU25 most probably would be better to roll-back the cut on MaxHIts.
The argument is that at PU15 the combinatorial is much smaller.
Of course we need to quantify. In absence of a PU15 MC we need to evaluate the risk

slava77 · 2015-10-03T00:44:10Z

+1

for #11492 bb306e1

changes are in line with the description of the PR
jenkins tests pass and show quite a few differences as expected from a change in tracking
higher stat tests were done in CMSSW_7_6_X_2015-09-29-0900
- see notes for MC and 2015B/C data (FEDs 434 439 still operational) above
- a 5K run of 256630 JetHT data (434 439 are out) shows the gaps in coverage are now filled, most visibly in the tobtec iterations; the plots are very similar to those shown in the DPG/POG meeting

A better solution for seeding or candidate building aware of the non-operational components would be better.

cmsbuild · 2015-10-03T00:46:22Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_6_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @smuzaffar

slava77 · 2015-10-03T00:47:46Z

I've seen tob-tec fakes introduce problems at analysis level in run1 data.
These were showing up significantly in "fat" high-et jets (dijet mass ~3 TeV), coming from the backgrounds (with some undesired spectrum distortion): not something that we'd be able to see in relvals.
Since then, the tobtec iteration was cleaned up quite a bit; so, the fakes are less of an issue.

davidlange6 · 2015-10-04T12:22:04Z

+1

add triplets using TOB and TIB4 for redundancy in TobTec and PIxelLess seeding steps respectively. reduce chi2-cut in MuonInOut

cmsbuild added this to the Next CMSSW_7_6_X milestone Sep 25, 2015

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending comparison-pending labels Sep 25, 2015

VinInn changed the title ~~add triplets using TOB4 for redundancy~~ add triplets using TOB4 for redundancy in toltec seeding step Sep 25, 2015

cmsbuild added tests-started and removed tests-pending labels Sep 25, 2015

cmsbuild added tests-approved and removed tests-started labels Sep 25, 2015

cmsbuild added comparison-available and removed comparison-pending labels Sep 25, 2015

VinInn changed the title ~~add triplets using TOB4 for redundancy in toltec seeding step~~ add triplets using TOB4, TIB4 and PIX1 for redundancy in tobtec, PIxelLess and MixedTriplet seeding step respectively. reduce chi2-cut in MuonInOut Sep 29, 2015

cmsbuild added dqm-pending analysis-pending db-pending tests-pending comparison-pending and removed comparison-available tests-approved labels Sep 29, 2015

add triplets using TOB4 for redundancy

f4ffdd7

cmsbuild added tests-approved and removed tests-started labels Sep 30, 2015

cmsbuild added comparison-available and removed comparison-pending labels Sep 30, 2015

cmsbuild added reconstruction-approved fully-signed and removed pending-signatures reconstruction-pending labels Oct 3, 2015

cmsbuild added orp-approved and removed orp-pending labels Oct 4, 2015

cmsbuild added a commit that referenced this pull request Oct 4, 2015

Merge pull request #11492 from VinInn/TOB4

59c43be

add triplets using TOB and TIB4 for redundancy in TobTec and PIxelLess seeding steps respectively. reduce chi2-cut in MuonInOut

cmsbuild merged commit 59c43be into cms-sw:CMSSW_7_6_X Oct 4, 2015

slava77 mentioned this pull request Oct 5, 2015

remove outlier rejection from mergedDuplicateTracks #11625

Closed

slava77 mentioned this pull request Oct 16, 2015

Tracking Reco: tuning of outlier rejection, duplicate merging, pairSeeding #11725

Closed

VinInn deleted the TOB4 branch January 30, 2016 11:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add triplets using TOB and TIB4 for redundancy in TobTec and PIxelLess seeding steps respectively. reduce chi2-cut in MuonInOut #11492

add triplets using TOB and TIB4 for redundancy in TobTec and PIxelLess seeding steps respectively. reduce chi2-cut in MuonInOut #11492

VinInn commented Sep 25, 2015

cmsbuild commented Sep 25, 2015

VinInn commented Sep 25, 2015

slava77 commented Sep 25, 2015

cmsbuild commented Sep 25, 2015

cmsbuild commented Sep 25, 2015

cmsbuild commented Sep 25, 2015

cmsbuild commented Sep 29, 2015

VinInn commented Sep 29, 2015

cmsbuild commented Sep 30, 2015

cmsbuild commented Sep 30, 2015

makortel commented Oct 1, 2015

VinInn commented Oct 1, 2015

VinInn commented Oct 1, 2015

slava77 commented Oct 1, 2015

VinInn commented Oct 1, 2015

slava77 commented Oct 1, 2015

VinInn commented Oct 1, 2015

slava77 commented Oct 2, 2015

VinInn commented Oct 2, 2015

slava77 commented Oct 3, 2015

cmsbuild commented Oct 3, 2015

slava77 commented Oct 3, 2015

davidlange6 commented Oct 4, 2015

add triplets using TOB and TIB4 for redundancy in TobTec and PIxelLess seeding steps respectively. reduce chi2-cut in MuonInOut #11492

add triplets using TOB and TIB4 for redundancy in TobTec and PIxelLess seeding steps respectively. reduce chi2-cut in MuonInOut #11492

Conversation

VinInn commented Sep 25, 2015

cmsbuild commented Sep 25, 2015

VinInn commented Sep 25, 2015

slava77 commented Sep 25, 2015

cmsbuild commented Sep 25, 2015

cmsbuild commented Sep 25, 2015

cmsbuild commented Sep 25, 2015

cmsbuild commented Sep 29, 2015

VinInn commented Sep 29, 2015

cmsbuild commented Sep 30, 2015

cmsbuild commented Sep 30, 2015

makortel commented Oct 1, 2015

VinInn commented Oct 1, 2015

VinInn commented Oct 1, 2015

slava77 commented Oct 1, 2015

VinInn commented Oct 1, 2015

slava77 commented Oct 1, 2015

VinInn commented Oct 1, 2015

slava77 commented Oct 2, 2015

VinInn commented Oct 2, 2015

slava77 commented Oct 3, 2015

cmsbuild commented Oct 3, 2015

slava77 commented Oct 3, 2015

davidlange6 commented Oct 4, 2015