fix HGCAL Layer Cluster times in heterogeneous workflows at HLT #45838

AuroraPerego · 2024-08-30T08:07:14Z

PR description:

The times and timeErrors vectors used to store the rechits time were initialized with a size of 16, but then push_back was used to fill them leaving the first 16 entries as 0.
This PR removes the allocation of 16 elements in the constructor, using reserve() instead.
The change affects only heterogeneous workflows at the HLT.

PR validation:

Tested on wf 31834.492, the HGCAL Layer Clusters times are expected to change:

old -> pre fix
new -> post fix

FYI @rovere

cmsbuild · 2024-08-30T08:07:41Z

cms-bot internal usage

cmsbuild · 2024-08-30T08:09:37Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45838/41582

cmsbuild · 2024-08-30T08:09:59Z

A new Pull Request was created by @AuroraPerego for master.

It involves the following packages:

RecoLocalCalo/HGCalRecProducers (upgrade, reconstruction)

@cmsbuild, @jfernan2, @mandrenguyen, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks.
@apsallid, @bsunanda, @cseez, @edjtscott, @felicepantaleo, @hatakeyamak, @lecriste, @lgray, @missirol, @pfs, @rovere, @sameasy, @sethzenz, @vandreev11, @youyingli this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

rovere · 2024-08-30T12:28:26Z

thanks @AuroraPerego for the investigation and the fix.

rovere · 2024-08-30T12:28:32Z

@cmsbuild please test

cmsbuild · 2024-08-30T15:40:15Z

+1

Size: This PR adds an extra 20KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4551d6/41199/summary.html
COMMIT: 95e7d74
CMSSW: CMSSW_14_2_X_2024-08-29-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45838/41199/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3328315
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3328289
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 193 log files, 163 edm output root files, 44 DQM output files
TriggerResults: no differences found

rovere · 2024-09-02T07:52:58Z

RecoLocalCalo/HGCalRecProducers/plugins/HGCalLayerClustersFromSoAProducer.cc

+    for (unsigned int i = 0; i < clusters->size(); ++i) {
+      times[i].reserve(16);
+      timeErrors[i].reserve(16);
+    }


Would it make sense to embed this loop into the previous one at lines 69 to 77? In this way, the heuristic is not even necessary and the size can be reserved correctly (i.e. the maximum possible, eventually some rechits will be discarded from the timing computation).

rovere · 2024-09-02T09:48:34Z

@cmsbuild please test

rovere · 2024-09-02T09:48:52Z

The bot seems to be a little stuck...

cmsbuild · 2024-09-02T09:51:37Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45838/41597

felicepantaleo · 2024-10-30T15:03:38Z

test parameters:

enable = gpu
workflows_gpu = 31834.492
workflow_opts = -w upgrade
workflow_opts_gpu = -w upgrade

felicepantaleo · 2024-10-30T15:04:11Z

@cmsbuild please test

cmsbuild · 2024-10-30T18:12:42Z

-1

Failed Tests: RelVals-GPU RelVals-INPUT
Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4551d6/42479/summary.html
COMMIT: c733b85
CMSSW: CMSSW_14_2_X_2024-10-30-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/45838/42479/install.sh to create a dev area with all the needed externals and cmssw changes.

DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals-GPU

ValueError: Undefined workflows: 31834.492

RelVals-INPUT

2024.0000012024.000001_RunJetMET02024D_10k/step1_dasquery.log
2024.0010012024.001001_RunZeroBias2024D_10k/step1_dasquery.log
2024.1000012024.100001_RunJetMET02024C_10k/step1_dasquery.log

Expand to see more relval errors ...

2024.101001
2024.000001
2024.001001
2024.100001
2024.101001

Comparison Summary

Summary:

You potentially added 4 lines to the logs
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3569372
DQMHistoTests: Total failures: 425
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3568927
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
Checked 201 log files, 171 edm output root files, 46 DQM output files
TriggerResults: no differences found

felicepantaleo · 2024-11-01T06:39:49Z

test parameters:

enable = gpu
workflows_gpu = 31834.496
workflow_opts = -w upgrade
workflow_opts_gpu = -w upgrade

felicepantaleo · 2024-11-01T06:40:43Z

please test

cmsbuild · 2024-11-01T11:15:24Z

+1

Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4551d6/42527/summary.html
COMMIT: c733b85
CMSSW: CMSSW_14_2_X_2024-10-31-2300/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/45838/42527/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 3 lines from the logs
Reco comparison results: 5 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3569567
DQMHistoTests: Total failures: 422
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3569125
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
Checked 201 log files, 171 edm output root files, 46 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 25 differences found in the comparisons
DQMHistoTests: Total files compared: 8
DQMHistoTests: Total histograms compared: 154009
DQMHistoTests: Total failures: 710
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 153299
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 7 files compared)
Checked 29 log files, 35 edm output root files, 8 DQM output files
TriggerResults: no differences found

jfernan2 · 2024-11-03T21:38:55Z

+1

Moanwar · 2024-11-04T11:37:20Z

+Upgrade

cmsbuild · 2024-11-04T11:37:46Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2024-11-04T12:38:00Z

+1

cmsbuild added this to the CMSSW_14_2_X milestone Aug 30, 2024

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending upgrade-pending code-checks-pending labels Aug 30, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 30, 2024

cmsbuild added tests-started and removed tests-pending labels Aug 30, 2024

cmsbuild added tests-approved and removed tests-started labels Aug 30, 2024

rovere reviewed Sep 2, 2024

View reviewed changes

use reserve instead of defining the size in the constructor

c733b85

AuroraPerego force-pushed the fixResize_pre7 branch from 95e7d74 to c733b85 Compare September 2, 2024 08:56

cmsbuild added tests-started code-checks-pending and removed tests-approved code-checks-approved labels Sep 2, 2024

cmsbuild removed the code-checks-pending label Sep 2, 2024

cmsbuild added tests-rejected and removed tests-started labels Sep 26, 2024

cmsbuild added tests-started and removed tests-rejected labels Oct 30, 2024

cmsbuild added tests-rejected and removed tests-started labels Oct 30, 2024

cmsbuild added tests-started and removed tests-rejected labels Nov 1, 2024

cmsbuild added tests-approved and removed tests-started labels Nov 1, 2024

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Nov 3, 2024

cmsbuild added fully-signed upgrade-approved and removed pending-signatures upgrade-pending labels Nov 4, 2024

cmsbuild added orp-approved and removed orp-pending labels Nov 4, 2024

cmsbuild merged commit 669f2de into cms-sw:master Nov 4, 2024
15 checks passed

cmsbuild mentioned this pull request Nov 4, 2024

Fixing Wf Names for ParkingDouble* #46601

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix HGCAL Layer Cluster times in heterogeneous workflows at HLT #45838

fix HGCAL Layer Cluster times in heterogeneous workflows at HLT #45838

AuroraPerego commented Aug 30, 2024

cmsbuild commented Aug 30, 2024 •

edited

Loading

cmsbuild commented Aug 30, 2024

cmsbuild commented Aug 30, 2024

rovere commented Aug 30, 2024

rovere commented Aug 30, 2024

cmsbuild commented Aug 30, 2024

rovere Sep 2, 2024

rovere commented Sep 2, 2024

rovere commented Sep 2, 2024

cmsbuild commented Sep 2, 2024

felicepantaleo commented Oct 30, 2024

felicepantaleo commented Oct 30, 2024

cmsbuild commented Oct 30, 2024

felicepantaleo commented Nov 1, 2024

felicepantaleo commented Nov 1, 2024

cmsbuild commented Nov 1, 2024

jfernan2 commented Nov 3, 2024

Moanwar commented Nov 4, 2024

cmsbuild commented Nov 4, 2024

mandrenguyen commented Nov 4, 2024

fix HGCAL Layer Cluster times in heterogeneous workflows at HLT #45838

fix HGCAL Layer Cluster times in heterogeneous workflows at HLT #45838

Conversation

AuroraPerego commented Aug 30, 2024

PR description:

PR validation:

cmsbuild commented Aug 30, 2024 • edited Loading

cmsbuild commented Aug 30, 2024

cmsbuild commented Aug 30, 2024

rovere commented Aug 30, 2024

rovere commented Aug 30, 2024

cmsbuild commented Aug 30, 2024

Comparison Summary

rovere Sep 2, 2024

Choose a reason for hiding this comment

rovere commented Sep 2, 2024

rovere commented Sep 2, 2024

cmsbuild commented Sep 2, 2024

felicepantaleo commented Oct 30, 2024

felicepantaleo commented Oct 30, 2024

cmsbuild commented Oct 30, 2024

RelVals-GPU

RelVals-INPUT

Comparison Summary

felicepantaleo commented Nov 1, 2024

felicepantaleo commented Nov 1, 2024

cmsbuild commented Nov 1, 2024

Comparison Summary

GPU Comparison Summary

jfernan2 commented Nov 3, 2024

Moanwar commented Nov 4, 2024

cmsbuild commented Nov 4, 2024

mandrenguyen commented Nov 4, 2024

cmsbuild commented Aug 30, 2024 •

edited

Loading