[Failing Test]: beam_PerformanceTests_PubsubIOIT_Python_Streaming failing due to #25617

[Abacn]

What happened?

Tests failing. Error message in Dataflow UI (not shown in log)

Could not load main session. Inspect which external dependencies are used in the main module of your pipeline. Verify that corresponding packages are installed in the pipeline runtime environment and their installed versions match the versions used in pipeline submission environment. For more information, see: https://beam.apache.org/documentation/sdks/python-pipeline-dependencies/
Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py", line 117, in create_harness _load_main_session(semi_persistent_directory)
File "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py", line 342, in _load_main_session pickler.load_session(session_file)
File "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py", line 65, in load_session return desired_pickle_lib.load_session(file_path)
File "/usr/local/lib/python3.7/site-packages/apache_beam/internal/dill_pickler.py", line 313, in load_session return dill.load_session(file_path)
File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 368, in load_session module = unpickler.load()
File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 472, in load obj = StockUnpickler.load(self)
File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 462, in find_class return StockUnpickler.find_class(self, module, name) ModuleNotFoundError: No module named 'hamcrest'

Issue Failure

Failure: Test is flaky

Issue Priority

Priority: 1 (unhealthy code / failing or flaky postcommit so we cannot be sure the product is healthy)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[Abacn]

The tests worked for a long time. Is failing pipeline in #25617 too strict?

[Abacn]

CC: @psobot @tvalentyn

[psobot]

Hm, #25617 should only cause this pipeline to fail if --save_main_session is provided and the resulting session file cannot be unpickled, but in the test definition I don't see any use of --save_main_session (although I could be misunderstanding which flags get applied to which test runs).

[tvalentyn]

other integration tests get hamcrest through https://github.com/apache/beam/blob/master/sdks/python/scripts/run_integration_test.sh#L219 , not sure why PubsubIOIT doesn't, perhaps it's launched differently.

[Abacn]

Yes, load test / performance test is an executable python script (having if ... __main__ block). hamcrest is only used in assert locally so that specific test actually does not need to save main session.