Update 141X data GTs in autoCond

[perrotta]

Update online GTs in autoCond after the end of the 2024 data taking (to be used for tests and development):

• run3_hlt: 141X_dataRun3_HLT_frozen_v2
  • Same as 140X_dataRun3_HLT_v2 with snapshot at 2024-06-13 14:22:43 (UTC)
  • Remove not-needed records from the HLT GT, see https://cms-talk.web.cern.ch/t/call-for-hlt-gt-record-information-by-subsystems/48795
  • Add records for GEM dead and masked strips, see https://cms-talk.web.cern.ch/t/gem-request-for-including-gts-gemmaskedstrips-gemdeadstrips/55154
  • Diff with respect to previous version _v1 is here (unfrozen)
• run3_data_express : 141X_dataRun3_Express_frozen_v4
  • Same as 141X_dataRun3_Express_v4 but with snapshot at 2024-11-28 13:23:29 (UTC)
  • Add records for GEM dead and masked strips, see https://cms-talk.web.cern.ch/t/gem-request-for-including-gts-gemmaskedstrips-gemdeadstrips/55154
  • Diff with respect to previous version _v3 is here (unfrozen)
• run3_data_prompt : 141X_dataRun3_Prompt_frozen_v4
  • Same as 141X_dataRun3_Prompt_v4 but with snapshot at 2024-11-28 13:26:44 (UTC)
  • Add records for GEM dead and masked strips, see https://cms-talk.web.cern.ch/t/gem-request-for-including-gts-gemmaskedstrips-gemdeadstrips/55154
  • Diff with respect to previous version _v3 is here (unfrozen)

Also the offline data GT needed to be fixed:

• run3_data : 141X_dataRun3_v5
  • New offline HCAL response correction production tag with fixes for era 2024C and 2024E, see https://cms-talk.web.cern.ch/t/call-for-conditions-re-reco-of-2024-eras-c-d-e/42331/25
  • Add records for GEM dead and masked strips, see https://cms-talk.web.cern.ch/t/gem-request-for-including-gts-gemmaskedstrips-gemdeadstrips/55154
  • Diff with respect to previous version _v4 is here

Moreover, to avoid crashes in the unit tests because of the removed (now unused) records in the HLT GT, the commit 6198154 comments out the scal_dqm_sourceclient-live_cfg.py test from the ones run in the unit tests of DQM/Integration: as correctly reported in a comment here below, the SCAL FED has been removed from DAQ since the end of Run 2.

PR validation:

The removal of the not-needed records from the HLT GT was verified online as documented in CMSALCAFAST-94
Rely on these PR tests for a wider coverage

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46824/42829

[cmsbuild]

A new Pull Request was created by @perrotta for master.

It involves the following packages:

• Configuration/AlCa (alca)

@atpathak, @cmsbuild, @consuegs, @perrotta can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @fabiocos, @missirol, @mmusich, @rsreds, @tocheng, @yuanchao this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[perrotta]

please test

[cmsbuild]

-1

Failed Tests: UnitTests
Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1f6165/43137/summary.html
COMMIT: 6198154
CMSSW: CMSSW_15_0_X_2024-11-28-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46824/43137/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test TestDQMOnlineClient-scal_dqm_sourceclient had ERRORS

Comparison Summary

Summary:

• You potentially added 1 lines to the logs
• Reco comparison results: 6 differences found in the comparisons
• DQMHistoTests: Total files compared: 46
• DQMHistoTests: Total histograms compared: 3484682
• DQMHistoTests: Total failures: 438
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3484224
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
• Checked 202 log files, 172 edm output root files, 46 DQM output files
• TriggerResults: no differences found

[perrotta]

please test with #46742

[perrotta]

abort
(#46742 has nothing to do with the removal of the unused records...)

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46824/42833

• There are other open Pull requests which might conflict with changes you have proposed:
  • File DQM/Integration/python/clients/scal_dqm_sourceclient-live_cfg.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46824/42849

[cmsbuild]

Pull request #46824 was updated. @antoniovagnerini, @atpathak, @cmsbuild, @consuegs, @perrotta, @rseidita can you please check and sign again.

[perrotta]

please test

[cmsbuild]

+1

Size: This PR adds an extra 20KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1f6165/43164/summary.html
COMMIT: 23b9148
CMSSW: CMSSW_15_0_X_2024-11-29-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46824/43164/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 6 differences found in the comparisons
• DQMHistoTests: Total files compared: 46
• DQMHistoTests: Total histograms compared: 3484682
• DQMHistoTests: Total failures: 491
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3484171
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
• Checked 202 log files, 172 edm output root files, 46 DQM output files
• TriggerResults: found differences in 1 / 44 workflows

[perrotta]

+alca

• No changes were expected because of the GT updates (HCAL response corrections in the offline GT were only updated with a small fix for eras 2024C and 2024E; all other GT updates are kind of technical), and no changes were observed in the test outputs

[perrotta]

@cms-sw/dqm-l2 could you please either sign, or post here a comment about what you'd like to further investigate before doing it?

[antoniovagnerini]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @mandrenguyen, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[antoniovagnerini]

@cms-sw/dqm-l2 could you please either sign, or post here a comment about what you'd like to further investigate before doing it?

just to notify that we will not be able to carry out the usual P5 tests for this PR since the playback system on which we run our tests (managed by Point 5 sysadmins) are offline for the YETS.

[mandrenguyen]

+1

[perrotta]

@cms-sw/dqm-l2 could you please either sign, or post here a comment about what you'd like to further investigate before doing it?

just to notify that we will not be able to carry out the usual P5 tests for this PR since the playback system on which we run our tests (managed by Point 5 sysadmins) are offline for the YETS.

Thank you @antoniovagnerini
Well, the removal of the not-needed records from the HLT GT was already tested online by DQM, as documented in CMSALCAFAST-94. Besides that, for DQM this PR is just acting on the unit tests, basically commenting out an obsolete one: I don't think there is anything else to be checked at P5.
If you are happy with this you could also sign the two backports #46831 and #46832 (for AlCa I will just wait for the battery of IB tests to be concluded successfully before signing them)