Add preliminary GTs for 2023 MC #41947

francescobrivio · 2023-06-13T20:48:19Z

PR description:

This PR updates the GTs for the 2023 MC production to preliminary GTs, as detailed in this CMSTalk post.

Important Note:
These GTs do not yet contain all the conditions needed for the 2023 MC production as a few of them are still work in progress, but, since most of them are already available, we decided to start adding them to the release so they can be validated via the usual release validation.

GT differences:

phase1_2023_realistic: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/131X_mcRun3_2023_realistic_v5/131X_mcRun3_2023_realistic_v6
phase1_2023_cosmics: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/131X_mcRun3_2023cosmics_realistic_deco_v5/131X_mcRun3_2023cosmics_realistic_deco_v6

PR validation:

Successfully run:

runTheMatrix.py -l 12434.0,12434.7,12634.0 -j 8 --ibeos

Backport:

Not a backport.
Backports to 13_1_X and 13_0_X will be opened only after all the conditions are available.

francescobrivio · 2023-06-13T20:50:23Z

urgent

as discussed today at the ORP this should be included in 13_2_0_pre2, FYI @perrotta @rappoccio @cms-sw/ppd-l2

francescobrivio · 2023-06-13T20:50:44Z

@cmsbuild please test

cmsbuild · 2023-06-13T20:54:13Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41947/35919

This PR adds an extra 16KB to repository

cmsbuild · 2023-06-13T20:54:38Z

A new Pull Request was created by @francescobrivio for master.

It involves the following packages:

Configuration/AlCa (alca)

@tvami, @saumyaphor4252, @francescobrivio can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

missirol · 2023-06-13T22:07:47Z

@francescobrivio , I think we also need to update L1TUtmTriggerMenuForRelValForRun3 to use this latest Run-3 L1T menu (both in this PR and its backports down to 13_0_X)

cmssw/Configuration/AlCa/python/autoCondModifiers.py

Line 118 in 0717bc8

    
           L1TUtmTriggerMenuForRelValForRun3 =  ','.join( ['L1Menu_Collisions2023_v1_1_0-v2_xml' , "L1TUtmTriggerMenuRcd",           connectionString, "", "2023-05-02 12:00:00.000"] )

@cms-sw/l1-l2 @eyigitba @caruta , there are L1T-related errors in the addOnTests, see
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-89f8dd/33133/addOnTests/hlt_mc_GRun/step1.log

An exception of category 'L1TMuonEndCap' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 1
   [1] Running path 'L1TAnalyzerEndpath'
   [2] Prefetching for module L1TGlobalSummary/'L1TGlobalSummary'
   [3] Prefetching for module L1TGlobalProducer/'simGtStage2Digis'
   [4] Prefetching for module L1TMuonProducer/'simGmtStage2Digis'
   [5] Calling method for module L1TMuonEndCapTrackProducer/'simEmtfDigis'
Exception Message:
Trying to use EMTF pc_lut_version = 3, does not exist!
----- End Fatal Exception -------------------------------------------------

cmsbuild · 2023-06-13T22:44:41Z

-1

Failed Tests: RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-89f8dd/33133/summary.html
COMMIT: 7cb4006
CMSSW: CMSSW_13_2_X_2023-06-13-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41947/33133/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

12434.0

----- Begin Fatal Exception 14-Jun-2023 00:00:43 CEST-----------------------
An exception of category 'L1TMuonEndCap' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLTAnalyzerEndpath'
   [2] Prefetching for module L1TRawToDigi/'hltGtStage2Digis'
   [3] Prefetching for module RawDataCollectorByLabel/'rawDataCollector'
   [4] Prefetching for module L1TDigiToRaw/'gmtStage2Raw'
   [5] Calling method for module L1TMuonEndCapTrackProducer/'simEmtfDigis'
Exception Message:
Trying to use EMTF pc_lut_version = 3, does not exist!
----- End Fatal Exception -------------------------------------------------

12434.7

----- Begin Fatal Exception 14-Jun-2023 00:01:47 CEST-----------------------
An exception of category 'L1TMuonEndCap' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLTAnalyzerEndpath'
   [2] Prefetching for module L1TRawToDigi/'hltGtStage2Digis'
   [3] Prefetching for module RawDataCollectorByLabel/'rawDataCollector'
   [4] Prefetching for module L1TDigiToRaw/'gmtStage2Raw'
   [5] Calling method for module L1TMuonEndCapTrackProducer/'simEmtfDigis'
Exception Message:
Trying to use EMTF pc_lut_version = 3, does not exist!
----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

12434.012434.0_TTbar_14TeV+2023/step2_TTbar_14TeV+2023.log
12434.712434.7_TTbar_14TeV+2023_trackingMkFit/step2_TTbar_14TeV+2023_trackingMkFit.log
12634.012634.0_TTbar_14TeV+2023PU/step2_TTbar_14TeV+2023PU.log

AddOn Tests

hlt_mc_GRun

----- Begin Fatal Exception 13-Jun-2023 23:45:11 CEST-----------------------
An exception of category 'L1TMuonEndCap' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 1
   [1] Running path 'L1TAnalyzerEndpath'
   [2] Prefetching for module L1TGlobalSummary/'L1TGlobalSummary'
   [3] Prefetching for module L1TGlobalProducer/'simGtStage2Digis'
   [4] Prefetching for module L1TMuonProducer/'simGmtStage2Digis'
   [5] Calling method for module L1TMuonEndCapTrackProducer/'simEmtfDigis'
Exception Message:
Trying to use EMTF pc_lut_version = 3, does not exist!
----- End Fatal Exception -------------------------------------------------

hlt_mc_PIon

----- Begin Fatal Exception 13-Jun-2023 23:45:22 CEST-----------------------
An exception of category 'L1TMuonEndCap' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 3
   [1] Running path 'L1TAnalyzerEndpath'
   [2] Prefetching for module L1TGlobalSummary/'L1TGlobalSummary'
   [3] Prefetching for module L1TGlobalProducer/'simGtStage2Digis'
   [4] Prefetching for module L1TMuonProducer/'simGmtStage2Digis'
   [5] Calling method for module L1TMuonEndCapTrackProducer/'simEmtfDigis'
Exception Message:
Trying to use EMTF pc_lut_version = 3, does not exist!
----- End Fatal Exception -------------------------------------------------

hlt_mc_PRef

----- Begin Fatal Exception 13-Jun-2023 23:45:17 CEST-----------------------
An exception of category 'L1TMuonEndCap' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 3
   [1] Running path 'L1TAnalyzerEndpath'
   [2] Prefetching for module L1TGlobalSummary/'L1TGlobalSummary'
   [3] Prefetching for module L1TGlobalProducer/'simGtStage2Digis'
   [4] Prefetching for module L1TMuonProducer/'simGmtStage2Digis'
   [5] Calling method for module L1TMuonEndCapTrackProducer/'simEmtfDigis'
Exception Message:
Trying to use EMTF pc_lut_version = 3, does not exist!
----- End Fatal Exception -------------------------------------------------

francescobrivio · 2023-06-14T06:09:01Z

@francescobrivio , I think we also need to update L1TUtmTriggerMenuForRelValForRun3 to use this latest Run-3 L1T menu (both in this PR and its backports down to 13_0_X)

Ciao @missirol, but that is for data right? here i'm only touching MC GTs. We will update the data GTs in a separate PR if that's ok for you?

@cms-sw/l1-l2 @eyigitba @caruta , there are L1T-related errors in the addOnTests, see
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-89f8dd/33133/addOnTests/hlt_mc_GRun/step1.log

I have just re run 12434.0 locally and it was ok, while here it fails. I'm running the addOnTests locally also now.

missirol · 2023-06-14T06:41:25Z

but that is for data right? here i'm only touching MC GTs. We will update the data GTs in a separate PR if that's ok for you?

Okay. It can be done in a separate PR (plus backports), but I would just find it easier to be consistent and, in one go, update the L1T menu in all wfs that use a Run-3 HLT menu (the only reason it works now is that the new L1T menu happens to be compatible with the current Run-3 HLT menu).

The update of the L1T menu in the other wfs [*] would have to come soon (next few days), before we introduce the next HLT menu, because we know the latter will only be compatible with the latest L1T menu (i.e. "L1T-v1_2_0").

[*] By the way, this also applies to the 2024 MC GT, since we use the Run-3 HLT menu in those wfs.

mmusich · 2023-06-14T06:47:34Z

but I would just find it easier to be consistent and, in one go, update the L1T menu in all wfs that use a Run-3 HLT menu

then I would rather descope all the L1T+HLT related updates in a separate PR (data and MC alike), unless they critically depend on some of the other subdetector conditions.
The goal is of this PR is to start passing through regular release validation whatever other conditions based ingredient is ready (as the title says this is preliminary) for the 2023 MC production as soon as possible to avoid surprises later on (2022 docet).

eyigitba · 2023-06-14T07:17:09Z

Ah, this is my bad actually. I should have modified this pc_lut_version code for MC apparently. I can submit a PR to fix this quickly.

francescobrivio · 2023-06-14T07:44:14Z

Ah, this is my bad actually. I should have modified this pc_lut_version code for MC apparently. I can submit a PR to fix this quickly.

@eyigitba for my understanding: how is this goign to be fixed? Do you need a PR or a new tag? Because ideally this tag should be used in 13_0_X and 13_1_X and for the 2023 MC production...so if a new PR (and release) is needed, then it should be done ASAP!

eyigitba · 2023-06-14T07:57:04Z

Hi @francescobrivio , this will require a PR to be consistent. Technically we can do it by customizing a tag for just MC production, but I'm not sure if this is the best idea. If you want we can discuss this over zoom to understand which solution would be better.

francescobrivio · 2023-06-14T08:48:20Z

Ok I discussed with Efe and we are testing a fix for this locally.
If successful we will make PRs to all the needed release cycles, i.e. we will also need a new 13_0_X for MC production, FYI @perrotta @rappoccio @cms-sw/ppd-l2

francescobrivio · 2023-06-14T09:23:08Z

test parameters:

pull_request = Change EMTF Primitive Conversion LUT Assignment for MC #41954

francescobrivio · 2023-06-14T09:24:16Z

@cmsbuild please test

cmsbuild · 2023-06-14T15:20:27Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-89f8dd/33139/summary.html
COMMIT: 7cb4006
CMSSW: CMSSW_13_2_X_2023-06-13-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41947/33139/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

12834.0DAS Error

Comparison Summary

Summary:

You potentially added 122 lines to the logs
Reco comparison results: 5464 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3196062
DQMHistoTests: Total failures: 119643
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3076397
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.328 KiB( 47 files compared)
DQMHistoSizes: changed ( 12434.0,... ): -0.164 KiB L1T/L1TStage2uGT
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: found differences in 2 / 46 workflows

francescobrivio · 2023-06-14T15:50:36Z

I think we can ignore this:

Failed Tests: RelVals-INPUT

while this:

DQMHistoTests: Total failures: 119643

is expected I'd say, given that we have changed 54 tags in the 2023 MC realistic GT!
But I'll take a better look as soon as possible.

tvami · 2023-06-14T17:42:59Z

@cmsbuild, please test

cmsbuild · 2023-06-15T00:16:51Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-89f8dd/33164/summary.html
COMMIT: 7cb4006
CMSSW: CMSSW_13_2_X_2023-06-14-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41947/33164/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 92 lines to the logs
Reco comparison results: 5461 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3196062
DQMHistoTests: Total failures: 119640
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3076400
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.328 KiB( 47 files compared)
DQMHistoSizes: changed ( 12434.0,... ): -0.164 KiB L1T/L1TStage2uGT
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: found differences in 2 / 46 workflows

francescobrivio · 2023-06-15T07:59:30Z

+1

I'll sign this myself given that my @cms-sw/alca-l2 co-conveners have already signed the backports (see [13_1_X] Add preliminary GTs for 2023 MC and new L1T menu to relval/ideal/2024/run4 GTs and re-snapshot data GTS #41959 (comment))
Tests pass
Exprected differences (due to the update of the GT) are are observed in 2023 MC workflows

cmsbuild · 2023-06-15T07:59:57Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-06-15T08:07:48Z

+1

Differences in outputs are quite large, but I imagine that they were evaluated and deemed acceptable
Further inspection will be carried out with the CMSSW_13_2_0_pre2 relvals that will be produced as soon as the pre-release will be built

Add preliminary GTs for 2023 MC

7cb4006

cmsbuild added this to the CMSSW_13_2_X milestone Jun 13, 2023

cmsbuild added alca-pending pending-signatures tests-pending orp-pending code-checks-pending labels Jun 13, 2023

cmsbuild added tests-started urgent and removed tests-pending labels Jun 13, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 13, 2023

cmsbuild added tests-rejected and removed tests-started labels Jun 13, 2023

cmsbuild added the requires-external label Jun 14, 2023

francescobrivio mentioned this pull request Jun 14, 2023

Change EMTF Primitive Conversion LUT Assignment for MC #41954

Merged

cmsbuild added tests-started and removed tests-rejected labels Jun 14, 2023

cmsbuild added tests-rejected and removed tests-started labels Jun 14, 2023

cmsbuild added tests-started and removed tests-rejected labels Jun 14, 2023

cmsbuild added tests-approved and removed tests-started labels Jun 15, 2023

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Jun 15, 2023

cmsbuild added orp-approved and removed orp-pending labels Jun 15, 2023

cmsbuild merged commit a27963e into cms-sw:master Jun 15, 2023

cmsbuild mentioned this pull request Jun 15, 2023

GCC 11.4; CUDA 11.8; cleanup for cmsplatf/cmsos use cms-sw/cmsdist#8545

Merged

francescobrivio deleted the alca-partial_2023_MC_GTs branch June 16, 2023 07:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add preliminary GTs for 2023 MC #41947

Add preliminary GTs for 2023 MC #41947

francescobrivio commented Jun 13, 2023

francescobrivio commented Jun 13, 2023

francescobrivio commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

missirol commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

francescobrivio commented Jun 14, 2023

missirol commented Jun 14, 2023

mmusich commented Jun 14, 2023

eyigitba commented Jun 14, 2023

francescobrivio commented Jun 14, 2023

eyigitba commented Jun 14, 2023

francescobrivio commented Jun 14, 2023

francescobrivio commented Jun 14, 2023

francescobrivio commented Jun 14, 2023

cmsbuild commented Jun 14, 2023

francescobrivio commented Jun 14, 2023

tvami commented Jun 14, 2023

cmsbuild commented Jun 15, 2023

francescobrivio commented Jun 15, 2023

cmsbuild commented Jun 15, 2023

perrotta commented Jun 15, 2023

Add preliminary GTs for 2023 MC #41947

Add preliminary GTs for 2023 MC #41947

Conversation

francescobrivio commented Jun 13, 2023

PR description:

PR validation:

Backport:

francescobrivio commented Jun 13, 2023

francescobrivio commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

missirol commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

RelVals

RelVals-INPUT

AddOn Tests

francescobrivio commented Jun 14, 2023

missirol commented Jun 14, 2023

mmusich commented Jun 14, 2023

eyigitba commented Jun 14, 2023

francescobrivio commented Jun 14, 2023

eyigitba commented Jun 14, 2023

francescobrivio commented Jun 14, 2023

francescobrivio commented Jun 14, 2023

francescobrivio commented Jun 14, 2023

cmsbuild commented Jun 14, 2023

RelVals-INPUT

Comparison Summary

francescobrivio commented Jun 14, 2023

tvami commented Jun 14, 2023

cmsbuild commented Jun 15, 2023

Comparison Summary

francescobrivio commented Jun 15, 2023

cmsbuild commented Jun 15, 2023

perrotta commented Jun 15, 2023