Updating Data GT for 2015 Run2 EOY Re-Reco, RunII Simulation with new SIM geometries, and Upgrade GT with Pixel conditions #12664

diguida · 2015-12-03T23:30:00Z

Summary of Global Tag Changes

RunII Simulation

Ideal scenario: 76X_mcRun2_design_v13: As 76X_mcRun2_design_v12 with the following changes:
- added further labels for Castor SIM geometry
Asymptotic scenario: 76X_mcRun2_asymptotic_v13: As 76X_mcRun2_asymptotic_v12 with the following changes:
- added further labels for Castor SIM geometry
Startup scenario: 76X_mcRun2_startup_v12: As 76X_mcRun2_startup_v11 with the following changes:
- added further labels for Castor SIM geometry

RunI Data

HLT processing: 76X_dataRun1_HLT_frozen_v10: As 76X_dataRun1_HLT_frozen_v9 with the following changes:
- added Record for HCAL negative energy filter
Offline processing: 76X_dataRun1_v15: As 76X_dataRun1_v11 with the following changes:
- explicit tag differences:
  - updated Tracker alignment, surface deformations and APE for the full range of 2015 data taking
  - updated Lumi Section based beamspot for 2015B,C,D taking into account new Tracker alignment
  - updated DT,CSC and muon GPR alignment due to updated Tracker alignment
  - updated DT vdrift and tTrig constants
  - updated CSC timing constants
  - updated SiStrip bad channel and particle APV gain corrections
  - updated Hcal gains and response correction (taking into account HE radiation damage and HO calibration)
  - updated Castor local reco calibrations (Channel quality,Electronic map, Gains, GainWidths, Pedestals, PedestalWidths, QIEData, RecoParams, Saturation corrections)
  - updated Ecal channel status, IC, AdcToGeV, time calibrations
  - updated Ecal Barrel and Endcap alignment due to updated Tracker alignment
  - updated ES IC
  - updated centrality table with HFtowers label
  - updated JEC with AK4Calo label for pp reference run at 5TeV
- differences introduced by append IOVs:
  - new IOVs for L1T O2O
  - new IOVs for ECAL O2O
  - new IOVs for HCAL LUT (alternating tables for 0T and 3.8T)
  - new IOVs for SiStrip O2O pointing to prompt tags
  - new IOVs for RunInfo (up to run 263284)
  - new IOVs for CSC Crosstalk, Gains, Noise and Pedestals
  - new IOVs for Ecal bad channels, laser corrections, pedestals, and pulse shapes
  - new IOVs for Preshower alignment (alternating 0T and 3.8T geometries)
  - new IOVs for HCal channel quality
  - new IOVs for Pixel templates at 0T.

RunII Data

HLT processing: 76X_dataRun2_HLT_frozen_v10: As 76X_dataRun2_HLT_frozen_v9 with the following changes:
- added Record for HCAL negative energy filter
HLT HI processing: 76X_dataRun2_HLTHI_v1: As 76X_dataRun2_HLT_frozen_v10 with the following changes:
- added Tracking MVA selection for HI
- updated SiStrip pedestals with full fledged values for SiStrip Zero Suppression at HLT
- updated AK4Calo JEC for HLT with values for HI
Offline processing: 76X_dataRun2_v15: As 76X_dataRun2_v11 with the following changes:
- explicit tag differences:
  - updated Tracker alignment, surface deformations and APE for the full range of 2015 data taking
  - updated Lumi Section based beamspot for 2015B,C,D taking into account new Tracker alignment
  - updated DT,CSC and muon GPR alignment due to updated Tracker alignment
  - updated DT vdrift and tTrig constants
  - updated CSC timing constants
  - updated SiStrip bad channel and particle APV gain corrections
  - updated Hcal gains and response correction (taking into account HE radiation damage and HO calibration)
  - updated Castor local reco calibrations (Channel quality,Electronic map, Gains, GainWidths, Pedestals, PedestalWidths, QIEData, RecoParams, Saturation corrections)
  - updated Ecal channel status, IC, AdcToGeV, time calibrations
  - updated Ecal Barrel and Endcap alignment due to updated Tracker alignment
  - updated ES IC
  - updated centrality table with HFtowers label
  - updated JEC with AK4Calo label for pp reference run at 5TeV
- differences introduced by append IOVs:
  - new IOVs for L1T O2O
  - new IOVs for ECAL O2O
  - new IOVs for HCAL LUT (alternating tables for 0T and 3.8T)
  - new IOVs for SiStrip O2O pointing to prompt tags
  - new IOVs for RunInfo (up to run 263284)
  - new IOVs for CSC Crosstalk, Gains, Noise and Pedestals
  - new IOVs for Ecal bad channels, laser corrections, pedestals, and pulse shapes
  - new IOVs for Preshower alignment (alternating 0T and 3.8T geometries)
  - new IOVs for HCal channel quality
  - new IOVs for Pixel templates at 0T.

Upgrade simulation

2017 ideal scenario: 76X_upgrade2017_design_v8: As 76X_upgrade2017_design_v7 with the following changes:
- New Pixel SIM LA for Phase-I
- New Pixel LA for Phase-I
- New Pixel templates for Phase-I
- New Pixel SIM gains for Phase-I
- New Pixel GenErrors for Phase-I
- New Pixel gains for Phase-I
- New Pixel quality for Phase-I.

cmsbuild · 2015-12-03T23:30:23Z

A new Pull Request was created by @diguida (Salvatore Di Guida) for CMSSW_7_6_X.

It involves the following packages:

Configuration/AlCa

@diguida, @cerminar, @cmsbuild, @franzoni, @mmusich, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @Martin-Grunewald this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

Following commands in first line of a comment are recognized

+1|approve[d]|sign[ed]: L1/L2's to approve it
-1|reject[ed]: L1/L2's to reject it
assign <category>[,<category>[,...]]: L1/L2's to request signatures from other categories
unassign <category>[,<category>[,...]]: L1/L2's to remove signatures from other categories
hold: L1/all L2's/release manager to mark it as on hold
unhold: L1/user who put this PR on hold
merge: L1/release managers to merge this request
[@cmsbuild,] please test: L1/L2 and selected users to start jenkins tests
[@cmsbuild,] please test with cms-sw/cmsdist#<PR>: L1/L2 and selected users to start jenkins tests using externals from cmsdist

diguida · 2015-12-03T23:30:39Z

please test

cmsbuild · 2015-12-03T23:30:55Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/10127/console

cmsbuild · 2015-12-04T00:53:48Z

Pull request #12664 was updated. @diguida, @cerminar, @cmsbuild, @franzoni, @mmusich, @davidlange6 can you please check and sign again.

diguida · 2015-12-04T00:53:54Z

please test

cmsbuild · 2015-12-04T00:54:01Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/10128/console

cmsbuild · 2015-12-04T01:29:00Z

+1
Tested at: 4c6de4c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12664/10127/summary.html

cmsbuild · 2015-12-04T02:29:48Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12664/10127/summary.html

cmsbuild · 2015-12-04T02:51:10Z

-1
Tested at: 64fa813
When I ran the RelVals I found an error in the following worklfows:
4.22 step1

DAS Error

140.53 step1

DAS Error

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12664/10128/summary.html

franzoni · 2015-12-04T06:38:43Z

please test

cmsbuild · 2015-12-04T06:39:04Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/10131/console

cmsbuild · 2015-12-04T21:51:53Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12664/10150/summary.html

cmsbuild · 2015-12-04T22:04:28Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12664/10151/summary.html

cmsbuild · 2015-12-04T22:12:51Z

+1
Tested at: 18b80a7
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12664/10148/summary.html

diguida · 2015-12-05T08:26:36Z

@slava77 now it is done, I looked at all the GT one by one and amended the commits every time I was finding something non correct.
You can now look at the plots. I am doing the same checks on the bot results.

diguida · 2015-12-05T08:28:59Z

@slava77 BTW, if you can share the tools for plot comparison, I'd like to run some 2015 RelVal with the new GT and compare the results w.r.t. the clean IB.

slava77 · 2015-12-05T11:32:32Z

On 12/5/15 12:29 AM, Salvatore Di Guida wrote:

@slava77 https://github.com/slava77 BTW, if you can share the tools
for plot comparison, I'd like to run some 2015 RelVal with the new GT
and compare the results w.r.t. the clean IB.

instructions are available at
https://twiki.cern.ch/twiki/bin/viewauth/CMS/RecoIntegration#Running_matrix_tests
and the following several bullets.
jenkins runs the same comparisons with a bit different wrapper
and with some automation in place where it sets thresholds in relmon and
"alternative-comparisons"
(also a DQM diff).

—
Reply to this email directly or view it on GitHub
#12664 (comment).

slava77 · 2015-12-05T16:32:55Z

here are some notes based on a test in CMSSW_7_6_2.
All changes of physics objects are in data workflows. (no changes in MC).
Compared to the summary of the PR, I don't really see any numerical changes in run1 other than in the castor reco (maybe the updates were made to IOVs not covered by the tests).

run 1 data changes are only in castor reco (in 2010 and 2011 HI and in 2011 pp; no change in 2012 tests since castor was not in that data)
e.g. 2011A pp (wf 1000)
run2 data changes are in a similar set of places in 2015B, C, and D. A somewhat superficial summary of differences is:
- in HO energy (significantly higher, mainly in the outer wheels)
- in ES energy (noticeably higher)
- in ECAL hit time (smallish shuffle, probably better peak in EB)
- is CSC time (better centered around 0)
- in DT time (better centered around 0)
- in the offline PV constrained to BS: x and y errors (smaller by ~10-20%, there is also a small shift of about 10 microns in y in all B, C, and D tested workflows)
- generalTracks chi2 is a bit larger (chi2prob more flat closer to 1 but has a bigger bump at 0)

diguida · 2015-12-07T13:56:43Z

@slava77 thanks a lot for your detailed review. They all make sense to me, even if I would like to see in closest details the tracking changes (in principle they should improve the chi2). I will approve this and then post other plots I have started to look at (weekend taken by online operations and crappy home network).

diguida · 2015-12-07T13:56:52Z

+1

cmsbuild · 2015-12-07T13:57:42Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_6_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @Degano, @smuzaffar

davidlange6 · 2015-12-08T06:33:21Z

+1
(but did we get an answer about castor corrections for run1 given the large change?)

Updating Data GT for 2015 Run2 EOY Re-Reco

mmusich · 2015-12-08T11:54:30Z

@davidlange6 https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2130/1/1.html

boudoul · 2015-12-21T16:23:47Z

Hi @diguida , just a small request on my side : Could you make the tiltle of such PRs more explicit in the future ? I noticed that the upgrade simulation GT was also changed in this 'Updating Data GT for 2015 Run2 EOY Re-Reco" , I had hard time to find it ...(and you are also changing run1data and run2 simulation btw ... so the tiltle is really not explicit...) Thanks a lot.

mmusich · 2015-12-21T17:05:07Z

@boudoul a separate PR existed for a while: #12371, we decided to include that commit in this PR not to leave the upgrade GT behind. General comment: since we are trying to minimize the number of PRs for each release it's bound to happen again to have all the scenarios updated. Given the limited amount of characters available for the title we generally choose the most meaningful update. We can use (unless there are specific cases) a neutral "Updated Global Tags for all scenarios"

diguida · 2015-12-21T18:18:56Z

@boudoul @mmusich cutting and playing with the limit of characters I was able to provide a more meaningful title.
Hope you like it.

boudoul · 2015-12-21T18:23:53Z

I do, Thanks!

diguida force-pushed the GTs_for76_v4 branch from b01db8a to 4c6de4c Compare December 3, 2015 23:30

cmsbuild added this to the Next CMSSW_7_6_X milestone Dec 3, 2015

cmsbuild added alca-pending pending-signatures tests-pending orp-pending comparison-pending labels Dec 3, 2015

cmsbuild added tests-started and removed tests-pending labels Dec 3, 2015

diguida mentioned this pull request Dec 3, 2015

Updating Data GT for 2015 Run2 EOY Re-Reco #12636

Closed

diguida force-pushed the GTs_for76_v4 branch from 4c6de4c to 64fa813 Compare December 4, 2015 00:53

cmsbuild added tests-pending and removed tests-started labels Dec 4, 2015

cmsbuild added tests-started and removed tests-pending labels Dec 4, 2015

cmsbuild added comparison-available and removed comparison-pending labels Dec 4, 2015

cmsbuild added tests-rejected and removed tests-started labels Dec 4, 2015

cmsbuild added tests-approved and removed tests-started labels Dec 4, 2015

cmsbuild added comparison-available and removed comparison-pending labels Dec 4, 2015

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Dec 7, 2015

cmsbuild added orp-approved and removed orp-pending labels Dec 8, 2015

cmsbuild added a commit that referenced this pull request Dec 8, 2015

Merge pull request #12664 from diguida/GTs_for76_v4

f8ef215

Updating Data GT for 2015 Run2 EOY Re-Reco

cmsbuild merged commit f8ef215 into cms-sw:CMSSW_7_6_X Dec 8, 2015

diguida changed the title ~~Updating Data GT for 2015 Run2 EOY Re-Reco~~ Updating Data GT for 2015 Run2 EOY Re-Reco, RunII Simulation with new SIM geometries, and Upgrade GT with Pixel conditions Dec 21, 2015

diguida mentioned this pull request Jan 14, 2016

preshower planes calib conditions #12929

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating Data GT for 2015 Run2 EOY Re-Reco, RunII Simulation with new SIM geometries, and Upgrade GT with Pixel conditions #12664

Updating Data GT for 2015 Run2 EOY Re-Reco, RunII Simulation with new SIM geometries, and Upgrade GT with Pixel conditions #12664

diguida commented Dec 3, 2015

cmsbuild commented Dec 3, 2015

diguida commented Dec 3, 2015

cmsbuild commented Dec 3, 2015

cmsbuild commented Dec 4, 2015

diguida commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

franzoni commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

diguida commented Dec 5, 2015

diguida commented Dec 5, 2015

slava77 commented Dec 5, 2015

slava77 commented Dec 5, 2015

diguida commented Dec 7, 2015

diguida commented Dec 7, 2015

cmsbuild commented Dec 7, 2015

davidlange6 commented Dec 8, 2015

mmusich commented Dec 8, 2015

boudoul commented Dec 21, 2015

mmusich commented Dec 21, 2015

diguida commented Dec 21, 2015

boudoul commented Dec 21, 2015

Updating Data GT for 2015 Run2 EOY Re-Reco, RunII Simulation with new SIM geometries, and Upgrade GT with Pixel conditions #12664

Updating Data GT for 2015 Run2 EOY Re-Reco, RunII Simulation with new SIM geometries, and Upgrade GT with Pixel conditions #12664

Conversation

diguida commented Dec 3, 2015

Summary of Global Tag Changes

RunII Simulation

RunI Data

RunII Data

Upgrade simulation

cmsbuild commented Dec 3, 2015

diguida commented Dec 3, 2015

cmsbuild commented Dec 3, 2015

cmsbuild commented Dec 4, 2015

diguida commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

franzoni commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

cmsbuild commented Dec 4, 2015

diguida commented Dec 5, 2015

diguida commented Dec 5, 2015

slava77 commented Dec 5, 2015

slava77 commented Dec 5, 2015

diguida commented Dec 7, 2015

diguida commented Dec 7, 2015

cmsbuild commented Dec 7, 2015

davidlange6 commented Dec 8, 2015

mmusich commented Dec 8, 2015

boudoul commented Dec 21, 2015

mmusich commented Dec 21, 2015

diguida commented Dec 21, 2015

boudoul commented Dec 21, 2015