bug: Tap tests parameters are cached across test classes

[edgarrmondragon]

Singer SDK Version

0.31.0

Is this a regression?

• Yes

Python Version

NA

Bug scope

Taps (catalog, state, etc.)

Operating System

NA

Description

A regression was introduced in v0.29.0, specifically by #1752.

Before, it was possible to run multiple test suites for a tap, each with a different input catalog. This was useful if the developer wanted to test different sets of streams with different configs. For example, tap-postgres does this:

TapPostgresTestNOSQLALCHEMY = get_tap_test_class(
    tap_class=TapPostgres,
    config=NO_SQLALCHEMY_CONFIG,
    catalog="tests/resources/data.json",
    custom_suites=[custom_test_replication_key],
)


# creating testing instance for isolated table in postgres
TapPostgresTestSelectedColumnsOnly = get_tap_test_class(
    tap_class=TapPostgres,
    config=SAMPLE_CONFIG,
    catalog="tests/resources/data_selected_columns_only.json",
    custom_suites=[custom_test_selected_columns_only],
)

Seems to be related to class-level attributes used to define pytest parameterization.

---

This is causing tap-postgres tests to fail: https://github.com/MeltanoLabs/tap-postgres/actions/runs/5799561671/job/15719827945?pr=192

Code

No response