Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add is_column_selection() to expression meta, enhance expand_selector #16479

Merged
merged 6 commits into from
May 26, 2024

Conversation

alexander-beedie
Copy link
Collaborator

@alexander-beedie alexander-beedie commented May 24, 2024

Closes #16448.

Extends the existing expression meta/introspection methods with a new one that determines if a given expression exclusively selects columns. This includes selectors, bare/multi col, regex col, dtype col, use of exclude, and the positional top-level functions (first/last/nth). Aliased column selections are optionally allowed, but not by default.

This allows expand_selector to offer a new "strict=False" option such that it can additionally expand expressions like cs.starts_with("x").exclude("x!!"), which would previously raise an error.

Should cover the Great Tables use-case in the linked issue (@machow).

(Also: fixed some versionadded/changed tags as a trivial drive-by).

Example

New meta method accurately identifies all of the below:

import polars.selectors as cs
import polars as pl

for expr in (
    pl.col("colx"),
    pl.col("colx", "coly"),
    pl.col(pl.NUMERIC_DTYPES),
    cs.numeric(),
    cs.numeric().exclude("n"),
    cs.numeric() - cs.unsigned_integer(),
    pl.first(),
    pl.last(),
    pl.nth(2),
):
    assert expr.meta.is_column_selection()  # -> True

Enhanced expand_selector:

schema = {
    "id": pl.Int64,
    "desc": pl.String,
    "count": pl.UInt32,
    "value": pl.Float64,
}

Before

cs.expand_selector(schema, cs.numeric().exclude("id"))
# TypeError: expected a selector; found ...

After

cs.expand_selector(schema, cs.numeric().exclude("id"), strict=False)
# ('count', 'value')

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars labels May 24, 2024
@alexander-beedie alexander-beedie changed the title feat: Add is_column_selection() to expression meta methods feat: Add is_column_selection() to expression meta methods, enhance expand_selector May 24, 2024
@alexander-beedie alexander-beedie changed the title feat: Add is_column_selection() to expression meta methods, enhance expand_selector feat: Add is_column_selection() to expression meta, enhance expand_selector May 24, 2024
Copy link

codecov bot commented May 24, 2024

Codecov Report

Attention: Patch coverage is 95.00000% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 81.43%. Comparing base (019dfe8) to head (57325f2).

Files Patch % Lines
crates/polars-plan/src/logical_plan/format.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #16479      +/-   ##
==========================================
- Coverage   81.45%   81.43%   -0.02%     
==========================================
  Files        1409     1409              
  Lines      184961   184977      +16     
  Branches     2978     2978              
==========================================
- Hits       150656   150642      -14     
- Misses      33784    33814      +30     
  Partials      521      521              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ritchie46 ritchie46 merged commit cd04f3d into pola-rs:main May 26, 2024
32 checks passed
@alexander-beedie alexander-beedie deleted the exclude-selector branch May 26, 2024 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Following a selector with .exclude() is not considered a selector
2 participants