Implement framework modules for EventID-related tests #46790

fwyzard · 2024-11-24T22:36:01Z

PR description:

Implement additional framework modules used to test and validate the behaviour of the MPI-based modules developed in #32632:

EmptySourceFromEventIDs is a variation of the EmptySource that creates runs, lumis and events (containing no products) based on the list of EventIDs provided in the configuration;
edmtest::EventIDProducer reads the EventID from the current event and copies it into the Event as a data product;
edmtest::EventIDValidator reads the EventID from the current event and compares it to a data product read from the Event.

EmptySourceFromEventIDs can be used to test how a module reacts to arbitrary transitions.

EventIDProducer and EventIDValidator can be used to validate that an object produced in a given event is being read back in the same event.

PR validation:

The unit tests for the new modules pass.

fwyzard · 2024-11-24T22:36:11Z

please test

cmsbuild · 2024-11-24T22:36:25Z

cms-bot internal usage

cmsbuild · 2024-11-24T22:39:23Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46790/42774

cmsbuild · 2024-11-24T22:39:40Z

A new Pull Request was created by @fwyzard for master.

It involves the following packages:

FWCore/Modules (core)
FWCore/TestModules (core)

@Dr15Jones, @makortel, @smuzaffar can you please review it and eventually sign? Thanks.
@makortel, @missirol, @wddgit this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2024-11-25T01:18:17Z

-1

Failed Tests: UnitTests
Size: This PR adds an extra 20KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79ec69/43033/summary.html
COMMIT: 69cf0d9
CMSSW: CMSSW_15_0_X_2024-11-24-0000/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46790/43033/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test TestFWCoreModulesEventIDValidator had ERRORS

Comparison Summary

Summary:

You potentially added 1 lines to the logs
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3483722
DQMHistoTests: Total failures: 514
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3483188
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
Checked 202 log files, 172 edm output root files, 46 DQM output files
TriggerResults: found differences in 1 / 44 workflows

fwyzard · 2024-11-25T06:58:19Z

please test

cmsbuild · 2024-11-25T07:00:29Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46790/42775

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Modules/src/EmptySourceFromEventIDs.cc modified in PR(s): Extend CMSSW to a distributed application over MPI #32632
- File FWCore/Modules/test/BuildFile.xml modified in PR(s): Extend CMSSW to a distributed application over MPI #32632
- File FWCore/Modules/test/testEmptySourceFromEventIDs_cfg.py modified in PR(s): Extend CMSSW to a distributed application over MPI #32632
- File FWCore/TestModules/README.md modified in PR(s): Extend CMSSW to a distributed application over MPI #32632
- File FWCore/TestModules/plugins/BuildFile.xml modified in PR(s): Extend CMSSW to a distributed application over MPI #32632
- File FWCore/TestModules/plugins/EventIDProducer.cc modified in PR(s): Extend CMSSW to a distributed application over MPI #32632
- File FWCore/TestModules/plugins/EventIDValidator.cc modified in PR(s): Extend CMSSW to a distributed application over MPI #32632
- File FWCore/TestModules/test/BuildFile.xml modified in PR(s): Extend CMSSW to a distributed application over MPI #32632
- File FWCore/TestModules/test/testEventIDValidator_cfg.py modified in PR(s): Extend CMSSW to a distributed application over MPI #32632

cmsbuild · 2024-11-25T07:00:49Z

Pull request #46790 was updated. @Dr15Jones, @makortel, @smuzaffar can you please check and sign again.

cmsbuild · 2024-11-25T08:35:08Z

+1

Size: This PR adds an extra 20KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79ec69/43036/summary.html
COMMIT: 271511c
CMSSW: CMSSW_15_0_X_2024-11-24-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46790/43036/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 30 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3483722
DQMHistoTests: Total failures: 1948
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3481754
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
Checked 202 log files, 172 edm output root files, 46 DQM output files
TriggerResults: found differences in 1 / 44 workflows

cmsbuild · 2024-11-26T12:56:26Z

-1

Failed Tests: HeaderConsistency UnitTests
Size: This PR adds an extra 40KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79ec69/43083/summary.html
COMMIT: 339cf8b
CMSSW: CMSSW_15_0_X_2024-11-25-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46790/43083/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test TestFWCoreModulesEmptySourceFromEventIDs had ERRORS

Comparison Summary

Summary:

You potentially removed 1 lines from the logs
Reco comparison results: 36 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3483722
DQMHistoTests: Total failures: 2055
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3481647
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
Checked 202 log files, 172 edm output root files, 46 DQM output files
TriggerResults: found differences in 1 / 44 workflows

fwyzard · 2024-11-27T15:58:07Z

please test

fwyzard · 2024-11-27T15:58:58Z

@makortel I removed the code about generic products, it will go in a separate PR.

cmsbuild · 2024-11-27T17:34:14Z

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79ec69/43115/summary.html
COMMIT: b73d08a
CMSSW: CMSSW_15_0_X_2024-11-27-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46790/43115/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 2 lines from the logs
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3483722
DQMHistoTests: Total failures: 508
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3483194
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
Checked 202 log files, 172 edm output root files, 46 DQM output files
TriggerResults: no differences found

makortel · 2024-11-27T19:37:42Z

Comparison differences are related to #39803 and #46416

makortel · 2024-11-27T19:39:56Z

+core

Comments in #32632 (review) were addressed. I'll open an issue about the need to refactor IDGeneratorSourceBase.

cmsbuild · 2024-11-27T19:40:19Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @sextonkennedy, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2024-11-28T07:44:08Z

+1

fwyzard · 2024-12-10T17:05:22Z

type ngt

cmsbuild added this to the CMSSW_15_0_X milestone Nov 24, 2024

cmsbuild added core-pending pending-signatures orp-pending tests-started code-checks-pending labels Nov 24, 2024

fwyzard mentioned this pull request Nov 24, 2024

Extend CMSSW to a distributed application over MPI #32632

Open

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 24, 2024

cmsbuild added tests-rejected and removed tests-started labels Nov 25, 2024

fwyzard force-pushed the implement_EventID_tools branch from 69cf0d9 to 271511c Compare November 25, 2024 06:57

cmsbuild added tests-pending code-checks-pending and removed tests-rejected code-checks-approved labels Nov 25, 2024

cmsbuild added tests-started and removed tests-pending labels Nov 25, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 25, 2024

cmsbuild removed the tests-started label Nov 25, 2024

cmsbuild added tests-started and removed tests-pending labels Nov 26, 2024

cmsbuild added tests-rejected and removed tests-started labels Nov 26, 2024

fwyzard force-pushed the implement_EventID_tools branch from 339cf8b to b73d08a Compare November 27, 2024 15:55

cmsbuild added tests-started and removed tests-rejected labels Nov 27, 2024

cmsbuild added tests-approved and removed tests-started labels Nov 27, 2024

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Nov 27, 2024

makortel mentioned this pull request Nov 27, 2024

Class template edm::IDGeneratorSourceBase<T> needs some refactoring #46814

Open

fwyzard changed the title ~~Implement framework modules for MPI-related tests~~ Implement framework modules for EventID-related tests Nov 28, 2024

cmsbuild added orp-approved and removed orp-pending labels Nov 28, 2024

cmsbuild merged commit 81b3ca6 into cms-sw:master Nov 28, 2024
11 checks passed

fwyzard deleted the implement_EventID_tools branch November 28, 2024 07:45

cmsbuild mentioned this pull request Nov 28, 2024

Make ROOT 6.32 as default for 15.0.X cms-sw/cmsdist#9541

Merged

cmsbuild added the ngt label Dec 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement framework modules for EventID-related tests #46790

Implement framework modules for EventID-related tests #46790

fwyzard commented Nov 24, 2024 •

edited

Loading

fwyzard commented Nov 24, 2024

cmsbuild commented Nov 24, 2024 •

edited

Loading

cmsbuild commented Nov 24, 2024

cmsbuild commented Nov 24, 2024

cmsbuild commented Nov 25, 2024

fwyzard commented Nov 25, 2024

cmsbuild commented Nov 25, 2024

cmsbuild commented Nov 25, 2024

cmsbuild commented Nov 25, 2024

cmsbuild commented Nov 26, 2024

fwyzard commented Nov 27, 2024

fwyzard commented Nov 27, 2024 •

edited

Loading

cmsbuild commented Nov 27, 2024

makortel commented Nov 27, 2024

makortel commented Nov 27, 2024

cmsbuild commented Nov 27, 2024

mandrenguyen commented Nov 28, 2024

fwyzard commented Dec 10, 2024

Implement framework modules for EventID-related tests #46790

Implement framework modules for EventID-related tests #46790

Conversation

fwyzard commented Nov 24, 2024 • edited Loading

PR description:

PR validation:

fwyzard commented Nov 24, 2024

cmsbuild commented Nov 24, 2024 • edited Loading

cmsbuild commented Nov 24, 2024

cmsbuild commented Nov 24, 2024

cmsbuild commented Nov 25, 2024

Unit Tests

Comparison Summary

fwyzard commented Nov 25, 2024

cmsbuild commented Nov 25, 2024

cmsbuild commented Nov 25, 2024

cmsbuild commented Nov 25, 2024

Comparison Summary

cmsbuild commented Nov 26, 2024

Unit Tests

Comparison Summary

fwyzard commented Nov 27, 2024

fwyzard commented Nov 27, 2024 • edited Loading

cmsbuild commented Nov 27, 2024

Comparison Summary

makortel commented Nov 27, 2024

makortel commented Nov 27, 2024

cmsbuild commented Nov 27, 2024

mandrenguyen commented Nov 28, 2024

fwyzard commented Dec 10, 2024

fwyzard commented Nov 24, 2024 •

edited

Loading

cmsbuild commented Nov 24, 2024 •

edited

Loading

fwyzard commented Nov 27, 2024 •

edited

Loading