Phase-2 L1T: Consolidation of e/g object interface #43317

cerminar · 2023-11-17T16:14:01Z

PR description:

This PR addresses several issues with the interface of TkElectron and TkEm objects and the HW objects used by the Phase2 GT emulator:

add ID score to both the hardware and CMSSW electron objects;
add interface to retrieve object in GT hw format;
remove Ref. to Egamma standalone objects since they are not the actual ancestors due to architectural choice in L1T hardware. The ref is replaced by an edm::Ptr to the calorimeter primitive (different in barrel and endcap).
further cleanup of the TkElectron interface and of data members;
charge is now populated for all TkElectrons;
change charge definition for GT electrons in accordance with new convention;
use non-PV-based photon isolation in hardware objects;
eta and phi at vertex (from the track) are now used for the TkElectron coordinates.

NOTE: backward compatibility is maintained via IO rule in the reflex dictionary.

PR validation:

Successfully tested reading files produced with previous versions of the object data formats.

…o GT HW object

… match FW

…idate and cleanup TkElectron interface

…n pv-based isolation. Set charge of TkElectrons

cmsbuild · 2023-11-17T16:21:16Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43317/37769

This PR adds an extra 64KB to repository

cmsbuild · 2023-11-17T16:21:42Z

A new Pull Request was created by @cerminar for master.

It involves the following packages:

DataFormats/L1TCorrelator (l1, upgrade)
DataFormats/L1TParticleFlow (l1, upgrade)
L1Trigger/Phase2L1ParticleFlow (l1, upgrade)

@srimanob, @cmsbuild, @epalencia, @AdrianoDee, @aloeliger can you please review it and eventually sign? Thanks.
@missirol, @Martin-Grunewald, @rovere this is something you requested to watch as well.
@antoniovilela, @sextonkennedy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

epalencia · 2023-11-17T17:44:41Z

Please test

cmsbuild · 2023-11-17T19:09:42Z

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4d6ea/35931/summary.html
COMMIT: 8872f78
CMSSW: CMSSW_14_0_X_2023-11-17-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43317/35931/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@Prasant1993 Add Correct EGM Run3 electron NonIso MVA ID for EB #43275

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4d6ea/35931/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4d6ea/35931/git-merge-result

Unit Tests

I found 1 errors in the following unit tests:

---> test runtestRecoEgammaElectronIdentification had ERRORS

RelVals

136.8311

----- Begin Fatal Exception 17-Nov-2023 19:23:14 CET-----------------------
An exception of category 'MVA config failure: ' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=ElectronMVAValueMapProducer label='electronMVAValueMapProducer'
Exception Message:
Concerning ElectronMVAEstimatorRun2RunIIIWinter22NoIsoV1
Variable ele_hcalPFClusterIso not found in variable definition file!
----- End Fatal Exception -------------------------------------------------

136.7611

----- Begin Fatal Exception 17-Nov-2023 19:23:14 CET-----------------------
An exception of category 'MVA config failure: ' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=ElectronMVAValueMapProducer label='electronMVAValueMapProducer'
Exception Message:
Concerning ElectronMVAEstimatorRun2RunIIIWinter22NoIsoV1
Variable ele_hcalPFClusterIso not found in variable definition file!
----- End Fatal Exception -------------------------------------------------

136.88811

----- Begin Fatal Exception 17-Nov-2023 19:23:14 CET-----------------------
An exception of category 'MVA config failure: ' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=ElectronMVAValueMapProducer label='electronMVAValueMapProducer'
Exception Message:
Concerning ElectronMVAEstimatorRun2RunIIIWinter22NoIsoV1
Variable ele_hcalPFClusterIso not found in variable definition file!
----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

4.64.6_MinimumBias2010A/step2_MinimumBias2010A.log
136.72411136.72411_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log
136.72412136.72412_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log

Expand to see more relval errors ...

cerminar · 2023-11-20T13:17:32Z

Not sure what happens here with the tests. The workflows mentioned above are not Phase-2 ones. Also, why is #43275 added to the test setup?
Thanks

mmusich · 2023-11-20T13:18:40Z

Not sure what happens here with the tests.

was a transient failure with the IB...

mmusich · 2023-11-20T13:18:47Z

@cmsbuild, please test

cmsbuild · 2023-11-20T16:13:33Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4d6ea/35958/summary.html
COMMIT: 8872f78
CMSSW: CMSSW_14_0_X_2023-11-20-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43317/35958/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 49 lines from the logs
Reco comparison results: 129 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3363868
DQMHistoTests: Total failures: 1775
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3362071
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 214 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

aloeliger · 2023-11-22T13:59:39Z

DataFormats/L1TCorrelator/src/classes_def.xml

-  <class name="l1t::TkElectron" ClassVersion="6">
+  <class name="l1t::TkElectron" ClassVersion="8">
+   <version ClassVersion="8" checksum="3565138561"/>
+   <version ClassVersion="7" checksum="3055953431"/>


What is the intermediate version used here? This should likely only bump the version up to version 7.

f607562#diff-7e45568c4065f523b35c57d6bfe035db03f4de890578025508a74fb21343abb5

The history of commits indeed changes the format twice.

aloeliger · 2023-11-27T09:22:45Z

+l1

cerminar · 2023-12-01T11:09:58Z

Hello, this conversation looks very silent....any feedback?

srimanob · 2023-12-01T14:06:36Z

@cmsbuild please test

Sorry, I just have a chance to review. Let's re-trigger the test after 2 weeks.

cmsbuild · 2023-12-01T16:51:45Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4d6ea/36254/summary.html
COMMIT: 8872f78
CMSSW: CMSSW_14_0_X_2023-11-30-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43317/36254/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 76 lines from the logs
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3370032
DQMHistoTests: Total failures: 76
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3369934
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 214 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

srimanob · 2023-12-02T09:30:03Z

+Upgrade

cmsbuild · 2023-12-02T09:30:29Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

srimanob · 2023-12-02T09:31:21Z

Note to @aloeliger @epalencia

Note relate to this PR. It should be good if you have a chance to look on static check log, and maybe consider to fix.
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4d6ea/36254/llvm-analysis/runStaticChecks.log

Thx.

antoniovilela · 2023-12-04T12:38:03Z

+1

cerminar added 5 commits November 17, 2023 16:42

Add enum for binary word encoding type and method for direct access t…

f607562

…o GT HW object

Write ID score to FW object and change initialization of bdt score to…

ac3ed7e

… match FW

Remove Ref to Standalone object in favor of generic pointer to L1Cand…

91270ac

…idate and cleanup TkElectron interface

GT format: change charge convention, use eta and phi at vertex, use o…

988f771

…n pv-based isolation. Set charge of TkElectrons

cosmetics

8872f78

cerminar mentioned this pull request Nov 17, 2023

Consolidation of Phase-2 e/g object interface cms-l1t-offline/cmssw#1181

Merged

cmsbuild added this to the CMSSW_14_0_X milestone Nov 17, 2023

cmsbuild added l1-pending pending-signatures tests-pending orp-pending upgrade-pending code-checks-pending labels Nov 17, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 17, 2023

cmsbuild added tests-started and removed tests-pending labels Nov 17, 2023

cmsbuild added tests-rejected and removed tests-started labels Nov 17, 2023

cmsbuild added tests-started and removed tests-rejected labels Nov 20, 2023

cmsbuild removed the tests-started label Nov 20, 2023

cmsbuild added the tests-approved label Nov 20, 2023

aloeliger reviewed Nov 22, 2023

View reviewed changes

cmsbuild added l1-approved and removed l1-pending labels Nov 27, 2023

cmsbuild added tests-started and removed tests-approved labels Dec 1, 2023

cmsbuild added tests-approved and removed tests-started labels Dec 1, 2023

cmsbuild added fully-signed upgrade-approved and removed pending-signatures upgrade-pending labels Dec 2, 2023

cmsbuild added orp-approved and removed orp-pending labels Dec 4, 2023

cmsbuild merged commit 3f0f809 into cms-sw:master Dec 4, 2023

This was referenced Dec 4, 2023

Add recipe for pytorch (C++ interface only) cms-sw/cmsdist#8388

Merged

Fix deprecated-copy warnings in CondFormats/PhysicsToolsObjects and PhysicsTools/MVAComputer #43400

Merged

Phase2 Trigger Primitive generation for Ecal Barrel #42633

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Phase-2 L1T: Consolidation of e/g object interface #43317

Phase-2 L1T: Consolidation of e/g object interface #43317

cerminar commented Nov 17, 2023 •

edited

Loading

cmsbuild commented Nov 17, 2023

cmsbuild commented Nov 17, 2023

epalencia commented Nov 17, 2023

cmsbuild commented Nov 17, 2023

cerminar commented Nov 20, 2023

mmusich commented Nov 20, 2023

mmusich commented Nov 20, 2023

cmsbuild commented Nov 20, 2023

aloeliger Nov 22, 2023

cerminar Nov 22, 2023

aloeliger commented Nov 27, 2023

cerminar commented Dec 1, 2023

srimanob commented Dec 1, 2023

cmsbuild commented Dec 1, 2023

srimanob commented Dec 2, 2023

cmsbuild commented Dec 2, 2023

srimanob commented Dec 2, 2023

antoniovilela commented Dec 4, 2023

Phase-2 L1T: Consolidation of e/g object interface #43317

Phase-2 L1T: Consolidation of e/g object interface #43317

Conversation

cerminar commented Nov 17, 2023 • edited Loading

PR description:

PR validation:

cmsbuild commented Nov 17, 2023

cmsbuild commented Nov 17, 2023

epalencia commented Nov 17, 2023

cmsbuild commented Nov 17, 2023

Unit Tests

RelVals

RelVals-INPUT

cerminar commented Nov 20, 2023

mmusich commented Nov 20, 2023

mmusich commented Nov 20, 2023

cmsbuild commented Nov 20, 2023

Comparison Summary

aloeliger Nov 22, 2023

Choose a reason for hiding this comment

cerminar Nov 22, 2023

Choose a reason for hiding this comment

aloeliger commented Nov 27, 2023

cerminar commented Dec 1, 2023

srimanob commented Dec 1, 2023

cmsbuild commented Dec 1, 2023

Comparison Summary

srimanob commented Dec 2, 2023

cmsbuild commented Dec 2, 2023

srimanob commented Dec 2, 2023

antoniovilela commented Dec 4, 2023

cerminar commented Nov 17, 2023 •

edited

Loading