You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Lack of generalization across different sources, stores, providers. Ideally we would have a high level set of e2e tests that use the user API to test different combination of source, stores, providers and other infra config. An example of the problem here is https://github.com/feast-dev/feast/blob/master/sdk/python/tests/test_historical_retrieval.py#L313 where we have a bigquery test and a parquet test.
Related to the above problem is the fact that we aren't reusing our setup/teardown for tests. Ideally the setup/teardown would be externalized and reusable across different kinds of tests (testing online retrieval, historical retrieval, materialization). Even better if we can just do the setup/teardown once, and then run a range of tests on the staged infrastructure.
Ideally we would have a single "kitchen sink" use case that can be applied to all tests, removing the need for test authors to think about what kind of data they need to use
We also need a place to add smaller tests, which are closer to unit tests. An example of this is testing whether entity dataframe inferencing works Infer entity dataframe event timestamp column #1495. Its not scalable to keep adding conditional flags and rerun the whole test suite. Eventually our integration tests will take too long to complete.
Lack of parallelism. We can run many of these tests async if they are well isolated.
Smaller point: We need to test all the types that we support. Currently we're not testing fringe types with our integration tests.
The text was updated successfully, but these errors were encountered:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Our current test suite has the following problems
The text was updated successfully, but these errors were encountered: