customiser for alpaka patatrack pixel seeding for phase2 #45508

lguzzi · 2024-07-19T08:54:02Z

PR description:

this PR adds alpaka patatrack pixel tracking in the simplified menu using the common alpaka process modifier.
The hltPhase2PixelTracks are redefined to be produced by CAHitNtupletAlpakaPhase2. Full tracks are produced merging initialStep (building from alpaka patatrack) + highPtTripletStep (building as in legacy).

PR

Alpaka tracking is enabled running the HLT step with --procModifiers alpaka:

cmsDriver.py step2  -s L1P2GT,HLT:75e33 --conditions auto:phase2_realistic_T33 --datatier GEN-SIM-DIGI-RAW -n 100 --eventcontent FEVTDEBUGHLT --geometry Extended2026D110 --era Phase2C17I13M9 --filein  file:step2_DIGIL1.root  --fileout file:step2_HLT.root  --nThreads 5 --procModifiers alpaka

(previous steps can be copied from runTheMatrix.py -l 29634.0 -t 4 -j 10 --ibeos --dryRun)

Tracking performance are reported here for general tracks and here for pixel tracks.
When moving from the old customiser function to process modifiers I checked that the two give identical results (when disabling HCAL alpaka reconstruction).

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

not a backport

cmsbuild · 2024-07-19T08:54:28Z

cms-bot internal usage

cmsbuild · 2024-07-19T08:55:52Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45508/40976

cmsbuild · 2024-07-19T08:56:12Z

A new Pull Request was created by @lguzzi for master.

It involves the following packages:

HLTrigger/Configuration (hlt)

@Martin-Grunewald, @cmsbuild, @mmusich can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich, @silviodonato this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

Martin-Grunewald · 2024-07-19T09:12:03Z

HLTrigger/Configuration/python/customizer_AlpakaPixelTracking_Phase2_141X.py

+                backend = cms.untracked.string('')
+        )
+    )
+    process.siPixelClustersSoA = cms.EDProducer('SiPixelPhase2DigiToCluster@alpaka',


Modules run at HLT start their label with hlt - this should now also be done for Phase-2.

I changed the name only for the new collections added by the customiser in beb3cb2
There are several modules and sequences in the phase2 menu which do not follow the naming convention. This problem is known, it will be fixed in a future PR

Martin-Grunewald · 2024-07-19T09:12:45Z

HLTrigger/Configuration/python/customizer_AlpakaPixelTracking_Phase2_141X.py

+    process.HLTDoLocalStripSequence = cms.Sequence(
+        process.siPhase2Clusters
+    )
+    process.itLocalRecoSequence = cms.Sequence(


Seqences run at HLT start their label with HLT - please do so also for Phase-2.

There are several modules and sequences in the phase2 menu which do not follow the naming convention. This problem is known, it will be fixed in a future PR. This sequence in particular is already defined in https://github.com/cms-sw/cmssw/blob/0cbc4b00b1664496105480e9f561a29ba7c9a68b/HLTrigger/Configuration/python/HLT_75e33/sequences/itLocalRecoSequence_cfi.py

Mmmh, as long as the new instances are used only in the customization function why do you need to keep the naming used elsewhere? Aren't you replacing the sequence entirely?

the idea was to allow other POGs to use the customiser with minimal effort. Some paths include this sequence (for example

cmssw/HLTrigger/Configuration/python/HLT_75e33/paths/HLT_Mu37_Mu27_FromL1TkMuon_cfi.py

Line 19 in 0cbc4b0

+itLocalRecoSequence

, but it is true for other sequences as well), and using a new name for the sequence would not propagate the changes done in the customiser to these paths

Martin-Grunewald · 2024-07-19T09:13:52Z

HLTrigger/Configuration/python/customizer_AlpakaPixelTracking_Phase2_141X.py

+    process.HLT_AlpakaTrackingPath.insert(item=process.HLTTrackingV61Sequence   , index=len(process.HLT_AlpakaTrackingPath.moduleNames()))
+    process.HLT_AlpakaTrackingPath.insert(item=process.HLTEndSequence           , index=len(process.HLT_AlpakaTrackingPath.moduleNames()))
+
+    process.FEVTDEBUGHLTEventContent.outputCommands.append('keep *_siPixelClusters_*_reHLT')


Why is this specific processName reHLT hardwired here and below?

these were leftovers, removed them in 2147f3d

Martin-Grunewald · 2024-07-19T09:15:02Z

HLTrigger/Configuration/python/customizer_AlpakaPixelTracking_Phase2_141X.py

+    process.FEVTDEBUGHLTEventContent.outputCommands.append('keep *_siPixelRecHits_*_reHLT')
+    process.FEVTDEBUGHLTEventContent.outputCommands.append('keep *_hltPhase2PixelTracks_*_reHLT')
+    process.FEVTDEBUGHLTEventContent.outputCommands.append('keep *_generalTracks_*_reHLT')
+    process.FEVTDEBUGHLTEventContent.outputCommands.append('drop IntermediateHitDoublets_hltElePixelHitDoubletsL1Seeded__reHLT')            # remove warning


Syntax is incorrect - I suppose L1Seeded_*_reHLT is required here and below?

these were leftovers, removed them in 2147f3d

mmusich · 2024-07-19T12:01:00Z

HLTrigger/Configuration/python/customizer_AlpakaPixelTracking_Phase2_141X.py

+        process.siPhase2Clusters
+    )
+    process.itLocalRecoSequence = cms.Sequence(
+         process.HLTDoLocalStripSequence


Why do you call this Strips for phase2? Something containing OT would be more appropriate

Also I find it confusing you call an OT local reco sequence within itLocalRecoSequence (again the naming convention is not respected).

this sequence is already defined in https://github.com/cms-sw/cmssw/blob/0cbc4b00b1664496105480e9f561a29ba7c9a68b/HLTrigger/Configuration/python/HLT_75e33/sequences/HLTDoLocalPixelSequence_cfi.py . Its redefinition here is actually not needed, so I removed it from the customiser in 208d97d. The problem of phase2 naming convention will be addressed in a future PR

the same is true for https://github.com/cms-sw/cmssw/blob/0cbc4b00b1664496105480e9f561a29ba7c9a68b/HLTrigger/Configuration/python/HLT_75e33/sequences/itLocalRecoSequence_cfi.py, which calls siPhase2Clusters as well. I did not change the structure of already existing sequences because they are called by other paths.

The problem of phase2 naming convention will be addressed in a future PR

@lguzzi please take note of #45500 (comment) (the renaming was kindly taken care of by @AuroraPerego). Please cross-check if you have comments.

mmusich · 2024-07-19T12:20:36Z

assign heterogeneous

cmsbuild · 2024-07-19T12:20:58Z

New categories assigned: heterogeneous

@fwyzard,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

fwyzard · 2024-07-19T12:51:26Z

HLTrigger/Configuration/python/customizer_AlpakaPixelTracking_Phase2_141X.py

I wouldn't put 141X in the file name, as it is likely to last into future releases as well.

done in 517e816

fwyzard · 2024-07-19T12:54:17Z

I would prefer that

the individual modules and sequences be implemented in their own files under HLTrigger/Configuration/python/HLT_75e33/eventsetup/, HLTrigger/Configuration/python/HLT_75e33/modules/, etc.
the customisation be implemented using modifiers instead of customise functions, and the changes written in the same file that has the definition of the modules

fwyzard · 2024-07-19T12:55:50Z

@rovere @VourMa @SohamBhattacharya FYI

cmsbuild · 2024-07-19T13:33:31Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45508/40978

Found files with invalid states:
- HLTrigger/Configuration/python/customizer_AlpakaPixelTracking_Phase2_141X.py:
  - Added: e12a1b6
  - Modified: 2147f3d, beb3cb2, 208d97d
  - Deleted: 517e816

cmsbuild · 2024-07-19T13:33:56Z

Pull request #45508 was updated. @Martin-Grunewald, @cmsbuild, @fwyzard, @makortel, @mmusich can you please check and sign again.

mmusich · 2024-07-19T14:02:06Z

the customisation be implemented using modifiers instead of customise functions, and the changes written in the same file that has the definition of the modules

For my own understanding, what's the advantage of that (in the long term)? See also discussion tentatively started at #45427 (comment)

fwyzard · 2024-07-19T14:03:51Z

In general, keeping the customisation close to the module or sequence that it affects makes it easier to see what impact it has.

Modifiers are a way to do that.

cmsbuild · 2024-09-10T09:23:10Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45508/41721

cmsbuild · 2024-09-10T09:23:34Z

Pull request #45508 was updated. @Martin-Grunewald, @cmsbuild, @fwyzard, @makortel, @mmusich can you please check and sign again.

mmusich · 2024-09-10T09:23:58Z

@cmsbuild, please test

cmsbuild · 2024-09-10T13:59:37Z

+1

Size: This PR adds an extra 36KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4bd8b/41432/summary.html
COMMIT: b4e8c86
CMSSW: CMSSW_14_2_X_2024-09-09-2300/el8_amd64_gcc12
Additional Tests: HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45508/41432/install.sh to create a dev area with all the needed externals and cmssw changes.

HLT P2 Timing: chart

Comparison Summary

Summary:

You potentially added 23 lines to the logs
Reco comparison results: 73 differences found in the comparisons
DQMHistoTests: Total files compared: 45
DQMHistoTests: Total histograms compared: 3437830
DQMHistoTests: Total failures: 4592
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3433218
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
Checked 197 log files, 168 edm output root files, 45 DQM output files
TriggerResults: no differences found

mmusich · 2024-09-12T07:13:20Z

+hlt

re-sign

fwyzard · 2024-09-13T09:15:21Z

+heterogeneous

cmsbuild · 2024-09-13T09:15:42Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @mandrenguyen, @sextonkennedy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2024-09-13T11:55:19Z

+1

cmsbuild added this to the CMSSW_14_1_X milestone Jul 19, 2024

cmsbuild added hlt-pending pending-signatures tests-pending orp-pending code-checks-pending labels Jul 19, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 19, 2024

Martin-Grunewald reviewed Jul 19, 2024

View reviewed changes

mmusich reviewed Jul 19, 2024

View reviewed changes

cmsbuild added the heterogeneous-pending label Jul 19, 2024

fwyzard reviewed Jul 19, 2024

View reviewed changes

cmsbuild added code-checks-pending and removed code-checks-approved labels Jul 19, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 19, 2024

cmsbuild added the code-checks-pending label Sep 10, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 10, 2024

cmsbuild added tests-started and removed tests-pending labels Sep 10, 2024

cmsbuild added tests-approved and removed tests-started labels Sep 10, 2024

cmsbuild mentioned this pull request Sep 11, 2024

[TauHLT] Add Phase2 MuTau HLT #45976

Merged

cmsbuild added hlt-approved and removed hlt-pending labels Sep 12, 2024

cmsbuild added fully-signed heterogeneous-approved and removed pending-signatures heterogeneous-pending labels Sep 13, 2024

cmsbuild added orp-approved and removed orp-pending labels Sep 13, 2024

cmsbuild merged commit e5de2f7 into cms-sw:master Sep 13, 2024
13 checks passed

This was referenced Sep 13, 2024

Refactoring of TICL EnergyRegression and PID model #45821

Merged

Fix a compilation warning in TracksterLinkingbySkeletons in DBG_X #45999

Merged

VourMa mentioned this pull request Sep 19, 2024

[DO NOT MERGE] Single iteration Patatrack and LST in HLT SegmentLinking/cmssw#94

Closed

lguzzi mentioned this pull request Sep 19, 2024

[phase2] add missing ESProducers to timing menu for running alpaka tracking #46060

Merged

mmusich mentioned this pull request Dec 2, 2024

Addition of single iteration Patatrack and LST HLT configurations and workflows #46828

Merged

customiser for alpaka patatrack pixel seeding for phase2 #45508

customiser for alpaka patatrack pixel seeding for phase2 #45508

Conversation

lguzzi commented Jul 19, 2024 • edited Loading

PR description:

PR

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Jul 19, 2024 • edited Loading

cmsbuild commented Jul 19, 2024

cmsbuild commented Jul 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Martin-Grunewald Jul 19, 2024 • edited Loading

Choose a reason for hiding this comment

lguzzi Jul 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mmusich commented Jul 19, 2024

cmsbuild commented Jul 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fwyzard commented Jul 19, 2024

fwyzard commented Jul 19, 2024

cmsbuild commented Jul 19, 2024

cmsbuild commented Jul 19, 2024

mmusich commented Jul 19, 2024

fwyzard commented Jul 19, 2024

cmsbuild commented Sep 10, 2024

cmsbuild commented Sep 10, 2024

mmusich commented Sep 10, 2024

cmsbuild commented Sep 10, 2024

Comparison Summary

mmusich commented Sep 12, 2024

fwyzard commented Sep 13, 2024

cmsbuild commented Sep 13, 2024

mandrenguyen commented Sep 13, 2024

lguzzi commented Jul 19, 2024 •

edited

Loading

cmsbuild commented Jul 19, 2024 •

edited

Loading

Martin-Grunewald Jul 19, 2024 •

edited

Loading

lguzzi Jul 19, 2024 •

edited

Loading