Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add method to add analyzer rules to SessionContext #10849

Merged
merged 6 commits into from
Jun 22, 2024
Merged

feat: Add method to add analyzer rules to SessionContext #10849

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
6b63dcc
feat: Add method to add analyzer rules to SessionContext
pingsutw Jun 10, 2024
9a1eae8
Merge branch 'main' of https://github.com/apache/datafusion into 10846
pingsutw Jun 12, 2024
e134719
Add a test
pingsutw Jun 12, 2024
62c5ab3
Add analyze_plan
pingsutw Jun 12, 2024
925f2c6
update test
pingsutw Jun 21, 2024
c81b4cf
Merge remote-tracking branch 'apache/main' into 10846
alamb Jun 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions datafusion/core/src/execution/context/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ use url::Url;
pub use datafusion_execution::config::SessionConfig;
pub use datafusion_execution::TaskContext;
pub use datafusion_expr::execution_props::ExecutionProps;
use datafusion_optimizer::AnalyzerRule;

mod avro;
mod csv;
Expand Down Expand Up @@ -331,6 +332,15 @@ impl SessionContext {
self
}

/// Adds an analyzer rule to the `SessionState` in the current `SessionContext`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to have an examples, or doc test for the this method

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree -- we are tracking adding an example for how to use custom analyzer rules in #10855, so perhaps we can add the example as part of that ticket (i think @goldmedal said he may have some time to work on that eventually)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I have a WIP PR to improve these examples -- I hope to get it up for review sometime this weekend

pub fn add_analyzer_rule(
self,
analyzer_rule: Arc<dyn AnalyzerRule + Send + Sync>,
) -> Self {
self.state.write().add_analyzer_rule(analyzer_rule);
self
}

/// Registers an [`ObjectStore`] to be used with a specific URL prefix.
///
/// See [`RuntimeEnv::register_object_store`] for more details.
Expand Down
4 changes: 2 additions & 2 deletions datafusion/core/src/execution/session_state.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -387,9 +387,9 @@ impl SessionState {
/// Add `analyzer_rule` to the end of the list of
/// [`AnalyzerRule`]s used to rewrite queries.
pub fn add_analyzer_rule(
mut self,
&mut self,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think technically this is an API change as now the api takes a mut reference rather than self

However, I think the change is good as now add_analyzer_rule looks more like a standard mutation style api (that takes &mut self) rather than a builder style (self)

What do you think about adding an api to make things consistent? (we could do this as a separate PR)

    pub fn with_analyzer_rule(
      mut self, 
        analyzer_rule: Arc<dyn AnalyzerRule + Send + Sync>,
  ) -> Self {
..
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also update add_optimizer_rule add_physical_optimizer_rule? (mut self -> &mut self)

Also, do we need to add with_optimizer_rule and with_physical_optimizer_rule to make it consistent?
If so, I can do it in a separate PR.

analyzer_rule: Arc<dyn AnalyzerRule + Send + Sync>,
) -> Self {
) -> &Self {
self.analyzer.rules.push(analyzer_rule);
self
}
Expand Down