Backport fixes for gen weights in NanoAOD (#31680, #31982, #31946, #31939, #32071) #32105

kdlong · 2020-11-11T13:36:37Z

PR description:

Backport of #31680, #31982, #31946, #31939, #32071. Fixes to Gen weight storage in NanoAOD

PR validation:

Tests in progress

if this PR is a backport please specify the original PR and why you need to backport that PR:

#31680, #31982, #31946, #31939, #32071

Needed for Nano v8 production in 106X

Fix a few places where missing_weightgroup wasn't set Debug back off Formatting

Switch loops in nanoaod producer so that PDF sets are prioritized by the order in the file, not the order in the config. Now chooses the first PDF set in the file which is part of the list of "approved" sets in the config. Also adds additional sets to this list

cmsbuild · 2020-11-11T13:37:01Z

A new Pull Request was created by @kdlong (Kenneth Long) for CMSSW_10_6_X.

It involves the following packages:

PhysicsTools/NanoAOD

@cmsbuild, @santocch, @mariadalfonso, @gouskos, @fgolf can you please review it and eventually sign? Thanks.
@gpetruc, @peruzzim, @swertz this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

mariadalfonso · 2020-11-11T14:28:27Z

please test

cmsbuild · 2020-11-11T14:28:48Z

The tests are being triggered in jenkins.

CMSSW_10_6_X_2020-11-11-1100/slc7_amd64_gcc700: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10649/console Started: 2020/11/11 15:30

cmsbuild · 2020-11-11T14:33:57Z

-1

Tested at: a036b3c

CMSSW: CMSSW_10_6_X_2020-11-11-1100
SCRAM_ARCH: slc7_amd64_gcc700
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26eabf/10649/summary.html

I found follow errors while testing this PR

Failed tests: Build ClangBuild

Build:

I found compilation error when building:

>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/GenWeightsTableProducer.cc 
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/BTagSFProducer.cc 
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/VertexTableProducer.cc 
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/NanoAODOutputModule.cc 
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/GenWeightsTableProducer.cc: In member function 'void GenWeightsTableProducer::fillLHEPdfWeightTablesFromGenInfo({anonymous}::Counter*, const {anonymous}::DynamicWeightChoiceGenInfo*, double, const GenEventInfoProduct&, std::unique_ptr&, std::unique_ptr&, std::unique_ptr&, std::unique_ptr&) const':
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/GenWeightsTableProducer.cc:483:46: error: 'psWeightDocStr' was not declared in this scope
                             wPS.size() > 1 ? psWeightDocStr : "dummy PS weight (1.0) ",
                                              ^~~~~~~~~~~~~~
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/GenWeightsTableProducer.cc:483:46: note: suggested alternative: 'psWeightsDocStr'
                             wPS.size() > 1 ? psWeightDocStr : "dummy PS weight (1.0) ",
                                              ^~~~~~~~~~~~~~

Clang:

I found compilation error while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' scram build -k -j 32 COMPILER='llvm compile'

>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/LeptonJetVarProducer.cc 
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/GenJetTauTaggerProducer.cc 
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/PileupJetIDVarProducer.cc 
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/EGMEnergyVarProducer.cc 
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/VIDNestedWPBitmapProducer.cc 
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/GenWeightsTableProducer.cc:483:46: error: use of undeclared identifier 'psWeightDocStr'; did you mean 'psWeightsDocStr'?
                            wPS.size() > 1 ? psWeightDocStr : "dummy PS weight (1.0) ",
                                             ^~~~~~~~~~~~~~
                                             psWeightsDocStr
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2020-11-11-1100/src/PhysicsTools/NanoAOD/plugins/GenWeightsTableProducer.cc:469:17: note: 'psWeightsDocStr' declared here
    std::string psWeightsDocStr;

cmsbuild · 2020-11-11T14:33:58Z

Comparison job queued.

mariadalfonso · 2020-11-11T14:40:53Z

This is what you have in master that is not backported here
https://github.com/cms-sw/cmssw/pull/31982/files#diff-c04ed34b9f6204d83b2af1010ba7bd69d2a1a3b7534322cbdc506436a59228b4L485

cmsbuild · 2020-11-11T19:55:02Z

Pull request #32105 was updated. @cmsbuild, @santocch, @mariadalfonso, @gouskos, @fgolf can you please check and sign again.

mariadalfonso · 2020-11-11T20:18:29Z

please test

cmsbuild · 2020-11-11T20:18:51Z

The tests are being triggered in jenkins.

CMSSW_10_6_X_2020-11-11-1100/slc7_amd64_gcc700: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10668/console Started: 2020/11/11 21:20

cmsbuild · 2020-11-11T21:40:46Z

+1
Tested at: ca24fd0
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26eabf/10668/summary.html
CMSSW: CMSSW_10_6_X_2020-11-11-1100
SCRAM_ARCH: slc7_amd64_gcc700

cmsbuild · 2020-11-11T21:40:49Z

Comparison job queued.

cmsbuild · 2020-11-11T23:26:09Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26eabf/10668/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 3214708
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3214372
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 140 log files, 14 edm output root files, 34 DQM output files

mariadalfonso · 2020-11-12T12:49:16Z

+xpog

changes inline with master.
(Size increase of about 2.5% in MC due to the PDF change (33-->103 weights))

santocch · 2020-11-12T19:28:24Z

+1

cmsbuild · 2020-11-12T19:28:47Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2020-11-13T00:10:13Z

+1

kdlong and others added 11 commits November 11, 2020 13:45

Porting Andrea's MG26X fixes

0d5f8e9

Fix a few places where missing_weightgroup wasn't set Debug back off Formatting

Fix central weight and error from mg26xNew

c61d391

Fix PSWeights in LumiHeader to work w diff pythia format

cec8d77

Fix PSWeights in LumiHeader to work w diff pythia format

5794422

Add NLO PDF4LHC15

1e99d0b

Catch EW parameter weights for mW sample

3c055c7

Restore import print statement

3c3e08b

Clang format

9fe854a

Modify scale regex to catch NNLOPS weights

aca6fe2

code check fix

a036b3c

cmsbuild added this to the CMSSW_10_6_X milestone Nov 11, 2020

cmsbuild added analysis-pending comparison-pending orp-pending pending-signatures tests-pending xpog-pending labels Nov 11, 2020

mariadalfonso mentioned this pull request Nov 11, 2020

features for V8 nanoAOD cms-nanoAOD/cmssw#533

Closed

55 tasks

cmsbuild added tests-started and removed tests-pending labels Nov 11, 2020

cmsbuild added tests-rejected and removed tests-started labels Nov 11, 2020

Fix conflict

ca24fd0

cmsbuild added tests-pending and removed tests-rejected labels Nov 11, 2020

cmsbuild added tests-started and removed tests-pending labels Nov 11, 2020

cmsbuild added tests-approved and removed tests-started labels Nov 11, 2020

cmsbuild added comparison-available and removed comparison-pending labels Nov 11, 2020

cmsbuild added xpog-approved and removed xpog-pending labels Nov 12, 2020

cmsbuild added analysis-approved fully-signed and removed analysis-pending pending-signatures labels Nov 12, 2020

cmsbuild added orp-approved and removed orp-pending labels Nov 13, 2020

cmsbuild merged commit 53e7a7d into cms-sw:CMSSW_10_6_X Nov 13, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backport fixes for gen weights in NanoAOD (#31680, #31982, #31946, #31939, #32071) #32105

Backport fixes for gen weights in NanoAOD (#31680, #31982, #31946, #31939, #32071) #32105

kdlong commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

mariadalfonso commented Nov 11, 2020

cmsbuild commented Nov 11, 2020 •

edited

Loading

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

mariadalfonso commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

mariadalfonso commented Nov 11, 2020

cmsbuild commented Nov 11, 2020 •

edited

Loading

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

mariadalfonso commented Nov 12, 2020

santocch commented Nov 12, 2020

cmsbuild commented Nov 12, 2020

qliphy commented Nov 13, 2020

Backport fixes for gen weights in NanoAOD (#31680, #31982, #31946, #31939, #32071) #32105

Backport fixes for gen weights in NanoAOD (#31680, #31982, #31946, #31939, #32071) #32105

Conversation

kdlong commented Nov 11, 2020

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Nov 11, 2020

mariadalfonso commented Nov 11, 2020

cmsbuild commented Nov 11, 2020 • edited Loading

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

mariadalfonso commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

mariadalfonso commented Nov 11, 2020

cmsbuild commented Nov 11, 2020 • edited Loading

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

cmsbuild commented Nov 11, 2020

mariadalfonso commented Nov 12, 2020

santocch commented Nov 12, 2020

cmsbuild commented Nov 12, 2020

qliphy commented Nov 13, 2020

cmsbuild commented Nov 11, 2020 •

edited

Loading

cmsbuild commented Nov 11, 2020 •

edited

Loading