Fixes for phase1 workflow

[makortel]

This PR fixes issues in the phase1 workflow runTheMatrix.py --what upgrade -l 10000. The changes are mainly to recover from #10373, and to remove the remaining CASTOR modules in reconstruction sequence (I guessed from the existing removals that this is the intention for now).

I also intentionally "broke" the "displaced muon reconstruction" (#7531) for Phase1 by taking InOut iteration out of its merger (technically it still should work but the results bad). It should be enabled again after the Phase1 tracking sequences are migrated to the new classifiers+mergers of #10373 (but I want to see the 76X-Phase1 tracking validated against 62X_SLHC Phase1 tracking before that).

Together with #11609 (GT update) this PR gets the reco step of upgrade wf 10000.0 to technically work on a DIGI step2.root produced with the earlier default GT (75X_upgrade2017_design_v1), but when using the same GT consistently, the reco step still fails. This failure is addressed separately in #11624.

Tested in 760pre6, no changes expected in Run1/2 workflows.

@rovere @VinInn @boudoul @venturia

[cmsbuild]

A new Pull Request was created by @makortel (Matti Kortelainen) for CMSSW_7_6_X.

Fixes for phase1 workflow

It involves the following packages:

RecoTracker/FinalTrackSelectors
RecoTracker/IterativeTracking
SLHCUpgradeSimulations/Configuration

@cmsbuild, @cvuosalo, @civanch, @mdhildreth, @slava77 can you please review it and eventually sign? Thanks.
@ghellwig, @GiacomoSguazzoni, @rovere, @VinInn, @Martin-Grunewald, @mschrode, @istaslis, @gpetruc, @cerati, @dgulhan this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@Degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

[boudoul]

FYI @mark-grimes
and big thanks to @makortel

[slava77]

@makortel
what's the meaning of "I also intentionally "broke" the "displaced muon reconstruction" (#7531) by taking InOut iteration out of its merger." ?
Is it really not working?
If so (pp MC/data processing has a regression in performance to make phase1 setup to work), at this point this PR should not be integrated in 76X until the run2 RECO is updated together consistently.

[makortel]

@slava77 It means that for --customise SLHCUpgradeSimulations/Configuration/combinedCustoms.cust_2017 I took muonSeededTracksInOut out of the merger of "displaced muon reconstruction", so it does not do what is intended (sorry for being sloppy, maybe also "broken" was a too strong word; reason was that I didn't want to spend effort in trying to mix the pre-#10373 classifiers and post-#10373 mergers, when the pre-#10373 classifiers will eventually be transformed to post-#10373 classifiers).

The change is done in phase1TkCustoms.customise_Reco()
https://github.com/cms-sw/cmssw/pull/11621/files#diff-7279af3f15fff92ce9c75462d0651681R406
As I wrote in the description, there should be no change in Run1/2 configurations.

[slava77]

@cmsbuild please test

Matti, thank you for the explanation.
Sorry, I wasn't paying enough attention to the full message and started typing the comment without reading to the end.

[cmsbuild]

The tests are being triggered in jenkins.

[cmsbuild]

-1
Tested at: cce3c34
The relvals timed out after 2 hours.

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11621/8590/summary.html

[slava77]

@cmsbuild please test
maybe better this time

[cmsbuild]

The tests are being triggered in jenkins.

[cmsbuild]

+1
Tested at: cce3c34
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11621/8593/summary.html

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11621/8593/summary.html

[civanch]

+1

[cvuosalo]

+1

For #11621 cce3c34

Fixes for Phase 1 upgrade workflow. Along with #11609 and #11624, this PR allows workflow 10000.0 to successfully complete the steps through RECO, but it still fails in the Harvesting step.

There should be no change in monitored quantities.

The code changes are satisfactory, and Jenkins tests against baseline CMSSW_7_6_X_2015-10-01-2300 show no significant differences, as expected. A test of workflow 10000.0 shows that it fails in baseline CMSSW_7_6_0_pre6 in the RECO step, but it successfully completes RECO (but not Harvesting) with this PR plus PRs #11609 and #11624.

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_6_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @smuzaffar

[davidlange6]

+1