Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

avoid holding memory between events in Multi5x5ClusterProducer #46756

Merged
merged 2 commits into from
Nov 22, 2024
Merged

avoid holding memory between events in Multi5x5ClusterProducer #46756

merged 2 commits into from
Nov 22, 2024

Conversation

Dr15Jones
Copy link
Contributor

@Dr15Jones Dr15Jones commented Nov 21, 2024 •
edited
Loading

PR description:

  • modernized Multi5x5ClusterProducer
    • added fillDescriptions
  • reworked Multi5x5ClusterAlgo
    • removed use of member data just for temporary storage
    • moved code doing related things into new functions/classes
    • use more modern C++ constructs

The holding and then clearing of memory across events was found by a prototype Alloc Monitor service.

NOTE: the only reason I could not change Multi5x5ClusterProducer to be a global module was the use of PositionCalc in Multi5x5ClusterAlgo.

PR validation:

Code compiles and running a very simple RECO job succeeded.

This is purely a refactoring so should not affect any physics results.

@Dr15Jones
Do not hold memory between events in Multi5x5ClusterAlgo
3f213a8 
- removed member variables which were just used as temporary storage
- refactored variables which must be updated together to be in a class
- modernized C++ syntax used
@Dr15Jones
Copy link
Contributor Author

@makortel FYI

@cmsbuild cmsbuild added this to the CMSSW_14_2_X milestone Nov 21, 2024
@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 21, 2024 •
edited
Loading

cms-bot internal usage

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46756/42735

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @Dr15Jones for master.

It involves the following packages:

  • RecoEcal/EgammaClusterAlgos (reconstruction)
  • RecoEcal/EgammaClusterProducers (reconstruction)

@cmsbuild, @jfernan2, @mandrenguyen can you please review it and eventually sign? Thanks.
@Prasant1993, @ReyerBand, @Sam-Harper, @a-kapoor, @afiqaize, @argiro, @jainshilpi, @lgray, @missirol, @ram1123, @rchatter, @rovere, @sameasy, @sobhatta, @thomreis, @valsdav, @varuns23, @wang0jin this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@Dr15Jones
Copy link
Contributor Author

please test

@makortel makortel mentioned this pull request Nov 21, 2024
Closed
@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2da7e6/42995/summary.html
COMMIT: f55a935
CMSSW: CMSSW_14_2_X_2024-11-21-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46756/42995/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

@Dr15Jones
Modernize Multi5x5ClusterProducer
c867bab 
- removed unnecessary member data
- avoid unnecessary allocations
- use new framework syntax
- added fillDescriptions
@Dr15Jones Dr15Jones force-pushed the allocMulti5x5ClusterAlgo branch from f55a935 to c867bab Compare November 21, 2024 22:20
@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46756/42738

@cmsbuild
Copy link
Contributor

Pull request #46756 was updated. @cmsbuild, @jfernan2, @mandrenguyen can you please check and sign again.

@Dr15Jones
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2da7e6/43001/summary.html
COMMIT: c867bab
CMSSW: CMSSW_14_2_X_2024-11-21-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46756/43001/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

@jfernan2
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @sextonkennedy, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

@antoniovilela
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 5e0fbce into cms-sw:master Nov 22, 2024
11 checks passed
@Dr15Jones
Copy link
Contributor Author

@jfernan2 looking at the results of the PR test, it looks like this module is producing a non-negligible number of difference over the original code, e.g.

https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_14_2_X_2024-11-21-1100+2da7e6/65771/validateJR/1000.0_RunMinBias2011A/all_RECO_step2_inAOD/c_recoPreshowerClusters_multi5x5SuperClustersWithPreshower_preshowerXClusters_RECO_obj_eta.png

and

https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_14_2_X_2024-11-21-1100+2da7e6/65771/9.0_Higgs200ChargedTaus/EcalClusterV_EcalSuperClusters.html

My expectations were that the code changes should have given identical physics.

@makortel
Copy link
Contributor

Should we revert the PR?

@cmsbuild cmsbuild mentioned this pull request Nov 22, 2024
Merged
@Dr15Jones
Copy link
Contributor Author

Dr15Jones commented Nov 22, 2024 •
edited
Loading

So I added a print statement into the module for both the original version and the new one. It appears the new one tends to have 1 more hit in clusters when things are different (below '<' denotes the new code and '>' the old code)

<  CaloCluster , algoID=4, CaloID: 2, E=28.5028, eta,phi=2.16838,0.610014, nhits=25
< ( 872440511, 1 ), ( 872440512, 1 ), ( 872440513, 1 ), ( 872440514, 1 ), ( 872440515, 1 ), ( 872440639, 1 ), ( 872440640, 1 ), ( 872440641, 1 ), ( 872440642, 1 ), ( 872440643, 1 ), ( 872440767, 1 ), ( 872440768, 1 ), ( 872440769, 1 ), ( 872440770, 1 ), ( 872440771, 1 ), ( 872440895, 1 ), ( 872440896, 1 ), ( 872440897, 1 ), ( 872440898, 1 ), ( 872440899, 1 ), ( 872441023, 1 ), ( 872441024, 1 ), ( 872441025, 1 ), ( 872441026, 1 ), ( 872441027, 1 ), 
<  CaloCluster , algoID=4, CaloID: 2, E=20.74, eta,phi=2.12595,0.736141, nhits=17
< ( 872440386, 1 ), ( 872440387, 1 ), ( 872440388, 1 ), ( 872440389, 1 ), ( 872440390, 1 ), ( 872440516, 1 ), ( 872440517, 1 ), ( 872440518, 1 ), ( 872440644, 1 ), ( 872440645, 1 ), ( 872440646, 1 ), ( 872440772, 1 ), ( 872440773, 1 ), ( 872440774, 1 ), ( 872440900, 1 ), ( 872440901, 1 ), ( 872440902, 1 ), 
---
>  CaloCluster , algoID=4, CaloID: 2, E=26.326, eta,phi=2.1625,0.614715, nhits=23
> ( 872440511, 1 ), ( 872440512, 1 ), ( 872440514, 1 ), ( 872440515, 1 ), ( 872440639, 1 ), ( 872440640, 1 ), ( 872440641, 1 ), ( 872440642, 1 ), ( 872440643, 1 ), ( 872440767, 1 ), ( 872440768, 1 ), ( 872440769, 1 ), ( 872440770, 1 ), ( 872440771, 1 ), ( 872440896, 1 ), ( 872440897, 1 ), ( 872440898, 1 ), ( 872440899, 1 ), ( 872441023, 1 ), ( 872441024, 1 ), ( 872441025, 1 ), ( 872441026, 1 ), ( 872441027, 1 ), 
>  CaloCluster , algoID=4, CaloID: 2, E=19.7252, eta,phi=2.1303,0.742665, nhits=16
> ( 872440386, 1 ), ( 872440387, 1 ), ( 872440388, 1 ), ( 872440389, 1 ), ( 872440390, 1 ), ( 872440516, 1 ), ( 872440517, 1 ), ( 872440518, 1 ), ( 872440644, 1 ), ( 872440645, 1 ), ( 872440646, 1 ), ( 872440772, 1 ), ( 872440773, 1 ), ( 872440774, 1 ), ( 872440901, 1 ), ( 872440902, 1 ), 
5576c5576

and

<  CaloCluster , algoID=4, CaloID: 2, E=39.4991, eta,phi=2.76572,-1.41006, nhits=21
< ( 872438180, 1 ), ( 872438181, 1 ), ( 872438182, 1 ), ( 872438183, 1 ), ( 872438308, 1 ), ( 872438309, 1 ), ( 872438310, 1 ), ( 872438311, 1 ), ( 872438435, 1 ), ( 872438436, 1 ), ( 872438437, 1 ), ( 872438438, 1 ), ( 872438439, 1 ), ( 872438564, 1 ), ( 872438565, 1 ), ( 872438566, 1 ), ( 872438567, 1 ), ( 872438692, 1 ), ( 872438693, 1 ), ( 872438694, 1 ), ( 872438695, 1 ), 
---
>  CaloCluster , algoID=4, CaloID: 2, E=34.8612, eta,phi=2.74065,-1.40841, nhits=20
> ( 872438180, 1 ), ( 872438181, 1 ), ( 872438182, 1 ), ( 872438183, 1 ), ( 872438308, 1 ), ( 872438309, 1 ), ( 872438310, 1 ), ( 872438435, 1 ), ( 872438436, 1 ), ( 872438437, 1 ), ( 872438438, 1 ), ( 872438439, 1 ), ( 872438564, 1 ), ( 872438565, 1 ), ( 872438566, 1 ), ( 872438567, 1 ), ( 872438692, 1 ), ( 872438693, 1 ), ( 872438694, 1 ), ( 872438695, 1 ), 
5740c5740

and

5887,5888d5886
<  CaloCluster , algoID=4, CaloID: 2, E=7.5786, eta,phi=-2.53928,3.05121, nhits=6
< ( 872419380, 1 ), ( 872419507, 1 ), ( 872419508, 1 ), ( 872419637, 1 ), ( 872419765, 1 ), ( 872419891, 1 ), 
5891,5892d5888
<  CaloCluster , algoID=4, CaloID: 2, E=9.02588, eta,phi=-2.88095,-0.0599388, nhits=5
< ( 872423217, 1 ), ( 872423219, 1 ), ( 872423345, 1 ), ( 872423346, 1 ), ( 872423475, 1 ), 
5900a5897,5898
>  CaloCluster , algoID=4, CaloID: 2, E=5.76277, eta,phi=-2.56595,3.04665, nhits=5
> ( 872419507, 1 ), ( 872419508, 1 ), ( 872419637, 1 ), ( 872419765, 1 ), ( 872419891, 1 ), 
5902a5901,5902
>  CaloCluster , algoID=4, CaloID: 2, E=6.99991, eta,phi=-2.88841,-0.0345329, nhits=4
> ( 872423217, 1 ), ( 872423219, 1 ), ( 872423346, 1 ), ( 872423475, 1 ), 
5977c5977

Looking at the entire CaloCluster list for the first diff, I see that the hit that is added in the new code (i.e. 872440513 ) is ALSO listed as being used by another cluster in the list

 CaloCluster , algoID=4, CaloID: 2, E=1.93585, eta,phi=2.24626,0.712407, nhits=3
( 872440258, 1 ), ( 872440259, 1 ), ( 872440513, 1 ),

@Dr15Jones Dr15Jones mentioned this pull request Nov 23, 2024
Merged
@Dr15Jones Dr15Jones deleted the allocMulti5x5ClusterAlgo branch November 25, 2024 14:24
@Dr15Jones Dr15Jones mentioned this pull request Nov 25, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
code-checks-approved fully-signed orp-approved reconstruction-approved tests-approved
Projects
None yet
Milestone
CMSSW_14_2_X
Development

Successfully merging this pull request may close these issues.
5 participants
@Dr15Jones @cmsbuild @jfernan2 @antoniovilela @makortel