Clean unused Phase-1 Pixel conditions in the Phase-2 autoCond key.

[mmusich]

Somewhat following up to this presentation I checked the current content of the latest and greatest phase2_realistic autoCond key in master (CMSSW_13_3_X at the time of writing)

https://github.com/cms-sw/cmssw/blob/6ec40408c72d87bf2073d4f2bb6694baf15f63b0/Configuration/AlCa/python/autoCond.py#L89C39-L89C63

and I see:

$ conddb list 131X_mcRun4_realistic_v6 | grep SiPixel
[2023-08-16 14:43:00,308] INFO: Connecting to pro [frontier://PromptProd/cms_conditions]
DQMXMLFileRcd                                           SiPixelDQMQTests                                                   DQMXMLFile_SiPixelDQM_v1_mc                                                 
SiPixelDetVOffRcd                                       -                                                                  SiPixelDetVOff_Ideal_RunBased_31X_v2_hlt                                    
SiPixelDynamicInefficiencyRcd                           -                                                                  SiPixelDynamicInefficiency_13TeV_v3_mc                                      
SiPixelFedCablingMapRcd                                 -                                                                  SiPixelFedCablingMap_mc                                                     
SiPixelGainCalibrationForHLTRcd                         -                                                                  SiPixelGainCalibrationForHLT_split_smeared_mc                               
SiPixelGainCalibrationOfflineRcd                        -                                                                  SiPixelGainCalibration_split_smeared_mc                                     
SiPixelGenErrorDBObjectRcd                              -                                                                  SiPixelGenErrorDBObject_phase2_IT_v6.1.5_25x100_v3_mc                       
SiPixelLorentzAngleRcd                                  -                                                                  SiPixelLorentzAngle_phase2_T15_v5_mc                                        
SiPixelLorentzAngleRcd                                  forWidth                                                           SiPixelLorentzAngle_phase2_T15_mc_forWidthEmpty                             
SiPixelLorentzAngleRcd                                  fromAlignment                                                      SiPixelLorentzAngle_phase2_T15_mc_forWidthEmpty                             
SiPixelLorentzAngleSimRcd                               -                                                                  SiPixelSimLorentzAngle_phase2_T15_v5_mc                                     
SiPixelQualityFromDbRcd                                 -                                                                  SiPixelQuality_phase2_ideal                                                 
SiPixelTemplateDBObjectRcd                              -                                                                  SiPixelTemplateDBObject_phase2_IT_v6.1.5_25x100_v3_mc                       
SiPixelTemplateDBObjectRcd                              0T                                                                 SiPixelTemplateDBObject_0T_v3_mc           

It looks to me that we can safely remove all the following records:

SiPixelDynamicInefficiencyRcd                           -                                                                  SiPixelDynamicInefficiency_13TeV_v3_mc                                      
SiPixelFedCablingMapRcd                                 -                                                                  SiPixelFedCablingMap_mc                                                     
SiPixelGainCalibrationForHLTRcd                         -                                                                  SiPixelGainCalibrationForHLT_split_smeared_mc                               
SiPixelGainCalibrationOfflineRcd                        -                                                                  SiPixelGainCalibration_split_smeared_mc                                                                  

this one

SiPixelTemplateDBObjectRcd                              0T                                                                 SiPixelTemplateDBObject_0T_v3_mc           

could be either removed (or updated, if there is any will to provide it).

[tvami]

Is there no plan to use these in the future? There should be some kind of gain calibration at one point in the detector, no?

[mmusich]

Is there no plan to use these in the future? There should be some kind of gain calibration at one point in the detector, no?

well, I am not sure the current data layout is appropriate at all, in any case why lugging these payloads (which are gynormous) when no-one currently consumes them!?

[saumyaphor4252]

I tested a candidate GT: 132X_mcRun4_realistic_TestPixelCleaning_v1 ( Diff )

with

• runTheMatrix.py -l 22034.0 -j 8 --ibeos --command "--conditions 132X_mcRun4_realistic_TestPixelCleaning_v1"
• runTheMatrix.py -l 24834.0 -j 8 --ibeos --command "--conditions 132X_mcRun4_realistic_TestPixelCleaning_v1"

but the workflows are failing with the following errors

----- Begin Fatal Exception 17-Aug-2023 06:48:09 CEST-----------------------
An exception of category 'GeometryMismatch' occurred while
   [0] Processing global begin Run run: 1
   [1] Prefetching for module L1FPGATrackProducer/'l1tTTTracksFromExtendedTrackletEmulation'
   [2] Prefetching for EventSetup module trklet::ProducerChannelAssignment/'ChannelAssignment'
   [3] Prefetching for EventSetup module tt::ProducerSetup/'TrackTriggerSetup'
   [4] Prefetching for EventSetup module TTStubAlgorithm_official_Phase2TrackerDigi_/''
   [5] Calling method for EventSetup module TrackerDigiGeometryESModule/'trackerGeometry'
Exception Message:
DetId mismatch between geometry (rawId=344462340) and alignments (rawId=344200196
----- End Fatal Exception -------------------------------------------------
Another exception was caught while trying to clean up files after the primary fatal exception.

----- Begin Fatal Exception 17-Aug-2023 07:29:01 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLTriggerFinalPath'
   [2] Prefetching for module TriggerSummaryProducerAOD/'hltTriggerSummaryAOD'
   [3] Prefetching for module EgammaHLTRecoEcalCandidateProducers/'hltEgammaCandidatesL1Seeded'
   [4] Prefetching for module PFECALSuperClusterProducer/'hltParticleFlowSuperClusterECALL1Seeded'
   [5] Prefetching for module CorrectedECALPFClusterProducer/'hltParticleFlowClusterECALL1Seeded'
   [6] Prefetching for module PFClusterProducer/'hltParticleFlowClusterECALUncorrectedL1Seeded'
   [7] Prefetching for module PFRecHitProducer/'hltParticleFlowRecHitECALL1Seeded'
   [8] Prefetching for module HLTEcalRecHitsInRegionsProducer/'hltRechitInRegionsECAL'
   [9] Prefetching for module L1TEGammaFilteredCollectionProducer/'hltL1TEGammaFilteredCollectionProducer'
   [10] Prefetching for module L1EGCrystalClusterEmulatorProducer/'l1tEGammaClusterEmuProducer'
   [11] Prefetching for module EcalEBTrigPrimProducer/'simEcalEBTriggerPrimitiveDigis'
   [12] Calling method for module MixingModule/'mix'
Exception Message:
No "SiPixelFedCablingMapRcd" record found in the EventSetup.

I am running in the IB CMSSW_13_3_X_2023-08-15-2300.

[mmusich]

@saumyaphor4252 thanks for the tests, however the first error you report

An exception of category 'GeometryMismatch' occurred while

it's not due to the cleaning but to your wrong coupling of geometry and Global Tag to run.
Indeed

22034.0 TTbar_14TeV_TuneCP5_2026D91_GenSimHLBeamSpot14+DigiTrigger_2026D91+RecoGlobal_2026D91+HARVESTGlobal_2026D91+ALCAPhase2_2026D91 [1]: cmsDriver.py TTbar_14TeV_TuneCP5_cfi  -s GEN,SIM -n 10 --conditions auto:phase2_realistic_T30 --beamspot HLLHC14TeV --datatier GEN-SIM --eventcontent FEVTDEBUG --geometry Extended2026D91 --era Phase2C17I13M9 --relval 9000,100 --conditions 132X_mcRun4_realistic_TestPixelCleaning_v1 

runs with auto:phase2_realistic_T30 which does customize several conditions (including the alignment) on top of the base GT.
The right testing setup is to edit the auto:phase2_realistic key in autoCond.py and then re-run the tests.
The second issue

An exception of category 'NoRecord' occurred while

looks genuine, but it's probably due to some missing clean-up (in principle the phase-2 workflow should not depend on a phase-1 condition). My advice is to keep SiPixelFedCablingMapRcd for the moment and remove it once the source code is patched.

[mmusich]

@emiglior

The second issue

An exception of category 'NoRecord' occurred while

looks genuine, but it's probably due to some missing clean-up (in principle the phase-2 workflow should not depend on a phase-1 condition

please clarify about this esConsumes call:

https://github.com/cms-sw/cmssw/blob/7769acb3589910f855d654cb693b400375beb602/SimTracker/SiPhase2Digitizer/plugins/PixelDigitizerAlgorithm.cc#L36

later on used at

https://github.com/cms-sw/cmssw/blob/7769acb3589910f855d654cb693b400375beb602/SimTracker/SiPhase2Digitizer/plugins/PixelDigitizerAlgorithm.cc#L292

As far as I can tell, this cannot be right (the FEDCabling map for phase-1 is used in the Phase-2 digitizer). Can you please comment why that piece of code is there?

[mmusich]

OK, so in order to make the GT clean-up work, we need some cleanup in cmssw first.
This commit mmusich/cmssw@6b962da +

diff --git a/Configuration/AlCa/python/autoCond.py b/Configuration/AlCa/python/autoCond.py
index efc62137819..d55fe5549d6 100644
--- a/Configuration/AlCa/python/autoCond.py
+++ b/Configuration/AlCa/python/autoCond.py
@@ -86,7 +86,7 @@ autoCond = {
     # GlobalTag for MC production with realistic conditions for Phase1 2024
     'phase1_2024_realistic'        : '132X_mcRun3_2024_realistic_v1',
     # GlobalTag for MC production with realistic conditions for Phase2
-    'phase2_realistic'             : '131X_mcRun4_realistic_v6'
+    'phase2_realistic'             : '132X_mcRun4_realistic_TestPixelCleaning_v1'
 }
 
 aliases = {

makes runTheMatrix.py -l 22034.0,24834.0 -t 4 -j 8 --ibeos run fine.

[mmusich]

@saumyaphor4252 now cms-sw/cmssw#42612 is merged.
You might want to go ahead with pushing 132X_mcRun4_realistic_TestPixelCleaning_v1 (or a better named GlobalTag) into autoCond.

[mmusich]

This issue was completed by a combination of cms-sw/cmssw#42612 and cms-sw/cmssw#42794

[saumyaphor4252]

Re-opening due to cms-sw/cmssw#42794 (comment)

[mmusich]

Re-opening due to cms-sw/cmssw#42794 (comment)

cms-sw/cmssw#42826 should solve.

[saumyaphor4252]

Fixed by cms-sw/cmssw#42826. See:

• Before: https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc11/CMSSW_13_3_X_2023-09-20-1100/unitTestLogs/Alignment/OfflineValidation#/2510-2510
• After: https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc11/CMSSW_13_3_X_2023-09-20-2300/unitTestLogs/Alignment/OfflineValidation#/2510-2510

[mmusich]

cms-sw/cmssw#42830 is still pending :)

[mmusich]

cms-sw/cmssw#42830 is merged and there weren't more failures, so I'll close the issue.