Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7 tests aren't running because they're misnamed #6316

Closed
MarcoGorelli opened this issue Jan 19, 2023 · 2 comments · Fixed by #6317
Closed

7 tests aren't running because they're misnamed #6316

MarcoGorelli opened this issue Jan 19, 2023 · 2 comments · Fixed by #6317

Comments

@MarcoGorelli
Copy link
Collaborator

MarcoGorelli commented Jan 19, 2023

Found via madforhooks

(venv) marcogorelli@DESKTOP-U8OKFP3:~/polars-dev/py-polars$ git ls-files | xargs python -m madforhooks.check_test_naming
tests/unit/io/test_parquet.py:236:0 found test function which does not start with 'test'
tests/unit/test_df.py:1539:0 found test function which does not start with 'test'
tests/unit/test_explode.py:57:0 found test function which does not start with 'test'
tests/unit/test_lists.py:489:0 found test function which does not start with 'test'
tests/unit/test_serde.py:16:0 found test function which does not start with 'test'
tests/unit/test_streaming.py:132:0 found test function which does not start with 'test'
tests/unit/test_struct.py:67:0 found test function which does not start with 'test'

This can be confirmed by running

(venv) marcogorelli@DESKTOP-U8OKFP3:~/polars-dev/py-polars$ pytest tests/ --cov=tests
====================================================== test session starts =======================================================
platform linux -- Python 3.10.6, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/marcogorelli/polars-dev/py-polars, configfile: pyproject.toml
plugins: hypothesis-6.62.1, cov-4.0.0
collected 1310 items                                                                                                             

tests/parametric/test_dataframe.py ...                                                                                     [  0%]
tests/parametric/test_lazyframe.py .                                                                                       [  0%]
tests/parametric/test_series.py ...                                                                                        [  0%]
tests/parametric/test_testing.py ........                                                                                  [  1%]
tests/slow/test_categorical.py .                                                                                           [  1%]
tests/slow/test_csv.py ...                                                                                                 [  1%]
tests/slow/test_overflow.py .                                                                                              [  1%]
tests/slow/test_parquet.py ...                                                                                             [  1%]
tests/slow/test_streaming.py ..                                                                                            [  1%]
tests/unit/test_aggregations.py .....                                                                                      [  2%]
tests/unit/test_api.py .....                                                                                               [  2%]
tests/unit/test_apply.py .................                                                                                 [  3%]
tests/unit/test_arithmetic.py ...                                                                                          [  4%]
tests/unit/test_binary.py .......                                                                                          [  4%]
tests/unit/test_build_info.py .                                                                                            [  4%]
tests/unit/test_categorical.py ...................                                                                         [  6%]
tests/unit/test_cfg.py ..........                                                                                          [  7%]
tests/unit/test_comparison.py .                                                                                            [  7%]
tests/unit/test_constructors.py ..................                                                                         [  8%]
tests/unit/test_context.py .                                                                                               [  8%]
tests/unit/test_cse.py ...                                                                                                 [  8%]
tests/unit/test_datatypes.py ............                                                                                  [  9%]
tests/unit/test_datelike.py ..........................................................................................s... [ 16%]
..............................                                                                                             [ 19%]
tests/unit/test_df.py .................................................................................................... [ 26%]
................................................                                                                           [ 30%]
tests/unit/test_empty.py ..                                                                                                [ 30%]
tests/unit/test_errors.py .........................                                                                        [ 32%]
tests/unit/test_explode.py ........                                                                                        [ 33%]
tests/unit/test_expr_multi_cols.py .....                                                                                   [ 33%]
tests/unit/test_exprs.py ..................................                                                                [ 36%]
tests/unit/test_filter.py .......                                                                                          [ 36%]
tests/unit/test_fmt.py ......                                                                                              [ 37%]
tests/unit/test_folds.py ...                                                                                               [ 37%]
tests/unit/test_functions.py ...........                                                                                   [ 38%]
tests/unit/test_groupby.py ...............                                                                                 [ 39%]
tests/unit/test_interop.py .........................................                                                       [ 42%]
tests/unit/test_joins.py ......................                                                                            [ 44%]
tests/unit/test_lazy.py .................................................................................................. [ 51%]
..............                                                                                                             [ 52%]
tests/unit/test_lists.py ...................................                                                               [ 55%]
tests/unit/test_meta.py ...                                                                                                [ 55%]
tests/unit/test_object.py ....                                                                                             [ 55%]
tests/unit/test_pivot.py ......................                                                                            [ 57%]
tests/unit/test_predicates.py .....                                                                                        [ 57%]
tests/unit/test_projections.py .........                                                                                   [ 58%]
tests/unit/test_queries.py ....................                                                                            [ 60%]
tests/unit/test_rolling.py ............                                                                                    [ 61%]
tests/unit/test_rows.py ...                                                                                                [ 61%]
tests/unit/test_schema.py .......................                                                                          [ 63%]
tests/unit/test_serde.py ....                                                                                              [ 63%]
tests/unit/test_series.py ................................................................................................ [ 70%]
........................................................................................                                   [ 77%]
tests/unit/test_show_versions.py .                                                                                         [ 77%]
tests/unit/test_sort.py .....................                                                                              [ 79%]
tests/unit/test_sql.py ....                                                                                                [ 79%]
tests/unit/test_streaming.py .....                                                                                         [ 79%]
tests/unit/test_strings.py .............                                                                                   [ 80%]
tests/unit/test_struct.py .........................................                                                        [ 83%]
tests/unit/test_testing.py ..................                                                                              [ 85%]
tests/unit/test_timezone.py ..                                                                                             [ 85%]
tests/unit/test_utf8.py ...                                                                                                [ 85%]
tests/unit/test_utils.py ..........                                                                                        [ 86%]
tests/unit/test_window.py ...............                                                                                  [ 87%]
tests/unit/io/test_avro.py ....                                                                                            [ 87%]
tests/unit/io/test_csv.py ........................................................................                         [ 93%]
tests/unit/io/test_delta.py ..........                                                                                     [ 94%]
tests/unit/io/test_excel.py ..                                                                                             [ 94%]
tests/unit/io/test_ipc.py ...........                                                                                      [ 95%]
tests/unit/io/test_json.py .....                                                                                           [ 95%]
tests/unit/io/test_lazy_csv.py .......                                                                                     [ 96%]
tests/unit/io/test_lazy_ipc.py ...                                                                                         [ 96%]
tests/unit/io/test_lazy_json.py ..                                                                                         [ 96%]
tests/unit/io/test_lazy_parquet.py .........                                                                               [ 97%]
tests/unit/io/test_other.py ....                                                                                           [ 97%]
tests/unit/io/test_parquet.py .............ss...............                                                               [ 99%]
tests/unit/io/test_pickle.py ..                                                                                            [ 99%]
tests/unit/io/test_pyarrow_dataset.py .                                                                                    [ 99%]
tests/unit/io/test_sql_db.py .                                                                                             [100%]

======================================================== warnings summary ========================================================
tests/slow/test_streaming.py::test_ooc_sort
  /home/marcogorelli/polars-dev/py-polars/tests/slow/test_streaming.py:26: DeprecationWarning: Series.shuffle will default to a random seed in a future version. Provide a value for the seed argument to silence this warning.
    df = s.shuffle().to_frame()

tests/unit/test_rows.py::test_row_tuple
  /home/marcogorelli/polars-dev/py-polars/tests/unit/test_rows.py:16: FutureWarning: Named rows will be changed from a namedtuple to a dictionary in the next breaking release.
    row = df.row(0, named=True)

tests/unit/test_rows.py::test_row_tuple
  /home/marcogorelli/polars-dev/py-polars/tests/unit/test_rows.py:26: FutureWarning: Named rows will be changed from a namedtuple to a dictionary in the next breaking release.
    row = df.row(by_predicate=pl.col("a") == "bar", named=True)

tests/unit/test_rows.py::test_rows
  /home/marcogorelli/polars-dev/py-polars/tests/unit/test_rows.py:63: FutureWarning: Named rows will be changed from a namedtuple to a dictionary in the next breaking release.
    rows = df.rows(named=True)

tests/unit/test_rows.py::test_iterrows
  /home/marcogorelli/polars-dev/py-polars/tests/unit/test_rows.py:91: FutureWarning: Named rows will be changed from a namedtuple to a dictionary in the next breaking release.
    row = next(it_named)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform linux, python 3.10.6-final-0 -----------
Name                                    Stmts   Miss Branch BrPart  Cover   Missing
-----------------------------------------------------------------------------------
tests/parametric/test_lazyframe.py         16      1      6      1    91%   54
tests/parametric/test_testing.py          105      3     42      3    96%   178, 182, 200
tests/slow/test_csv.py                     16      0      6      1    95%   17->exit
tests/slow/test_parquet.py                 29      0     12      3    93%   11->exit, 32->exit, 47->exit
tests/unit/conftest.py                     48      3      6      3    89%   70, 73, 76
tests/unit/io/test_ipc.py                  88      0     22      4    96%   34->exit, 43->exit, 102->exit, 140->exit
tests/unit/io/test_lazy_parquet.py         70      1     14      3    95%   139, 164->exit, 181->exit
tests/unit/io/test_parquet.py             183      9     40      0    96%   237-246
tests/unit/io/test_pyarrow_dataset.py      22      0     18      1    98%   23->exit
tests/unit/io/test_sql_db.py               20     10      4      0    50%   17-54
tests/unit/test_api.py                     67      0     30      1    99%   139->130
tests/unit/test_apply.py                   84      4     42     14    84%   22->exit, 33->exit, 38-41, 55->exit, 71, 73->exit, 91->exit, 96->exit, 105->exit, 134->exit, 147->exit, 154->exit, 204->exit, 232->exit, 246->exit, 268->exit
tests/unit/test_build_info.py               7      0      2      1    89%   8->exit
tests/unit/test_constructors.py           269      1     36      0    99%   336
tests/unit/test_datelike.py               610      6     58      2    99%   16, 1332->exit, 1857-1862
tests/unit/test_df.py                    1177     11    225      4    99%   677-678, 699-700, 723-724, 938->exit, 1054->exit, 1445-1452, 1540-1543, 1620->exit
tests/unit/test_errors.py                 126      0     64      1    99%   243->exit
tests/unit/test_explode.py                 48      5      0      0    90%   58-82
tests/unit/test_expr_multi_cols.py         18      0      2      1    95%   20->exit
tests/unit/test_folds.py                   18      0      8      4    85%   17->exit, 21->exit, 34->exit, 41->exit
tests/unit/test_lazy.py                   639      0     58     12    98%   68->exit, 184->exit, 745->exit, 752->exit, 759->exit, 766->exit, 775->exit, 784->exit, 795->exit, 810->exit, 1438->exit, 1448->exit
tests/unit/test_lists.py                  217      2     12      0    99%   490-497
tests/unit/test_object.py                  25      0      4      1    97%   66->exit
tests/unit/test_projections.py             45      0      2      1    98%   102->exit
tests/unit/test_schema.py                 101      0     18      1    99%   159->exit
tests/unit/test_serde.py                   24      3      2      0    88%   17-20
tests/unit/test_streaming.py               49      9     12      0    79%   133-165
tests/unit/test_struct.py                 231      6     24      1    96%   68-104, 276->exit
-----------------------------------------------------------------------------------
TOTAL                                    8715     74   1369     63    98%

51 files skipped due to complete coverage.

Required test coverage of 85.0% reached. Total coverage: 98.46%

which shows those tests as uncovered


These tests should be renamed to start with test_

@MarcoGorelli
Copy link
Collaborator Author

MarcoGorelli commented Jan 19, 2023

test_struct_unnesting actually fails

=========================================================================== FAILURES ============================================================================
_____________________________________________________________________ test_struct_unnesting _____________________________________________________________________

    def test_struct_unnesting() -> None:
        df = pl.DataFrame({"a": [1, 2]})
>       out = df.select(
            [
                pl.all().alias("a_original"),
                pl.col("a")
                .apply(lambda x: (x, x * 2, x % 2 == 0))
                .struct.rename_fields(["a", "a_squared", "mod2eq0"])
                .alias("foo"),
            ]
        ).unnest("foo")

tests/unit/test_struct.py:69: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
polars/internals/dataframe/frame.py:5597: in select
    self.lazy().select(exprs).collect(no_optimization=True)._df
polars/utils.py:394: in wrapper
    return fn(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <polars.LazyFrame object at 0x7F43919128C0>

    @deprecated_alias(allow_streaming="streaming")
    def collect(
        self,
        *,
        type_coercion: bool = True,
        predicate_pushdown: bool = True,
        projection_pushdown: bool = True,
        simplify_expression: bool = True,
        no_optimization: bool = False,
        slice_pushdown: bool = True,
        common_subplan_elimination: bool = True,
        streaming: bool = False,
    ) -> pli.DataFrame:
        """
        Collect into a DataFrame.
    
        Note: use :func:`fetch` if you want to run your query on the first `n` rows
        only. This can be a huge time saver in debugging queries.
    
        Parameters
        ----------
        type_coercion
            Do type coercion optimization.
        predicate_pushdown
            Do predicate pushdown optimization.
        projection_pushdown
            Do projection pushdown optimization.
        simplify_expression
            Run simplify expressions optimization.
        no_optimization
            Turn off (certain) optimizations.
        slice_pushdown
            Slice pushdown optimization.
        common_subplan_elimination
            Will try to cache branching subplans that occur on self-joins or unions.
        streaming
            Run parts of the query in a streaming fashion (this is in an alpha state)
    
        Returns
        -------
        DataFrame
    
        Examples
        --------
        >>> df = pl.DataFrame(
        ...     {
        ...         "a": ["a", "b", "a", "b", "b", "c"],
        ...         "b": [1, 2, 3, 4, 5, 6],
        ...         "c": [6, 5, 4, 3, 2, 1],
        ...     }
        ... ).lazy()
        >>> df.groupby("a", maintain_order=True).agg(pl.all().sum()).collect()
        shape: (3, 3)
        ┌─────┬─────┬─────┐
        │ a   ┆ b   ┆ c   │
        │ --- ┆ --- ┆ --- │
        │ str ┆ i64 ┆ i64 │
        ╞═════╪═════╪═════╡
        │ a   ┆ 4   ┆ 10  │
        │ b   ┆ 11  ┆ 10  │
        │ c   ┆ 6   ┆ 1   │
        └─────┴─────┴─────┘
    
        """
        if no_optimization:
            predicate_pushdown = False
            projection_pushdown = False
            slice_pushdown = False
            common_subplan_elimination = False
    
        if streaming:
            common_subplan_elimination = False
    
        ldf = self._ldf.optimization_toggle(
            type_coercion,
            predicate_pushdown,
            projection_pushdown,
            simplify_expression,
            slice_pushdown,
            common_subplan_elimination,
            streaming,
        )
>       return pli.wrap_df(ldf.collect())
E       exceptions.SchemaError: Series of dtype: Object("object") != Struct

polars/internals/lazyframe/frame.py:1168: SchemaError
==================================================================== short test summary info ====================================================================
FAILED tests/unit/test_struct.py::test_struct_unnesting - exceptions.SchemaError: Series of dtype: Object("object") != Struct

@MarcoGorelli
Copy link
Collaborator Author

looks like it started failing in polars 0.13.24

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant