[CHORE] Rust Logical plan skeleton

[xcharleslin]

Skeleton for new logical plan in Rust. Includes the following building blocks:

• LogicalPlan enum. This is the core logical plan object. It currently has just one variant, Source.
• SourceInfo enum, which contains information about what the data source is. Currently has one variant, FilesInfo, to represent file source.
• LogicalPlanBuilder, which wraps a LogicalPlan to:
  • Serve as the API surface for Python.
  • (later) Do resolution of unresolved names to field IDs.
• PyO3 bindings for LogicalPlanBuilder.

Next steps (separate PRs):

• Add Daft flag to switch between old and new logical plans.
• Have Daft create the new logical plan for file source operations. This will involve a new metadata resolution path.
• A lightweight physical plan translation in Rust.
• Have Daft be able to dispatch the translated physical plan, by hooking into the existing plan dispatching machinery.

[codecov]

Codecov Report

Merging #1192 (d032844) into main (bc11e57) will increase coverage by 0.00%.
Report is 1 commits behind head on main.
The diff coverage is n/a.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1192   +/-   ##
=======================================
  Coverage   88.40%   88.41%           
=======================================
  Files          55       55           
  Lines        5606     5617   +11     
=======================================
+ Hits         4956     4966   +10     
- Misses        650      651    +1     

see 1 file with indirect coverage changes

[clarkzinzow]

Probably don't need the underscore prefix to denote that the field is private, since struct fields are private by default and would only be made public with the pub plan qualifier.

[xcharleslin]

I needed to do it for clippy since the field is not yet used 😅

[clarkzinzow]

Nit: Since this is a creator for LogicalPlanBuilder, it would probably make more sense to have this as a from_source method.

We should also probably have FilesInfo::new() and Source::new() constructor methods for those structs, especially the latter which will always set empty fields that will be populated later.

Suggested change

	// Create a new LogicalPlanBuilder for a Source node.
	pub fn new_plan_from_source(filepaths: Vec<String>, schema: SchemaRef) -> LogicalPlanBuilder {
	let source_info = source_info::SourceInfo::FilesInfo(source_info::FilesInfo {
	filepaths,
	schema: schema.clone(),
	});
	let source_node = LogicalPlan::Source(ops::Source {
	schema,
	source_info: source_info.into(),
	filters: vec![], // Will be populated by plan optimizer.
	limit: None, // Will be populated by plan optimizer.
	});
	LogicalPlanBuilder {
	_plan: source_node.into(),
	}
	}
	impl LogicalPlanBuilder {
	// Create a new LogicalPlanBuilder for a Source node.
	pub fn from_source(filepaths: Vec<String>, schema: SchemaRef) -> Self {
	let source_info = source_info::SourceInfo::FilesInfo(source_info::FilesInfo::new(filepaths, schema);
	let source_node = LogicalPlan::Source(ops::Source::new(schema, source_info.into());
	Self {
	_plan: source_node.into(),
	}
	}

[xcharleslin]

Done! I also made from_source take a Source struct.

[clarkzinzow]

Could we expose the LogicalPlanBuilder directly via pyo3 rather than wrapping, similar to what we did for the ImageMode enum?

Daft/src/daft-core/src/datatypes/image_mode.rs

Lines 27 to 58 in bc11e57

	#[cfg_attr(feature = "python", pyclass)]
	pub enum ImageMode {
	L = 1,
	LA = 2,
	RGB = 3,
	RGBA = 4,
	L16 = 5,
	LA16 = 6,
	RGB16 = 7,
	RGBA16 = 8,
	RGB32F = 9,
	RGBA32F = 10,
	}
	#[cfg(feature = "python")]
	#[pymethods]
	impl ImageMode {
	/// Create an ImageMode from its string representation.
	///
	/// Args:
	/// mode: String representation of the mode. This is the same as the enum
	/// attribute name, e.g. ``ImageMode.from_mode_string("RGB")`` would
	/// return ``ImageMode.RGB``.
	#[staticmethod]
	pub fn from_mode_string(mode: &str) -> PyResult<Self> {
	Self::from_str(mode).map_err(|e| PyValueError::new_err(e.to_string()))
	}
	pub fn __str__(&self) -> PyResult<String> {
	Ok(self.to_string())
	}
	}

[xcharleslin]

Thank you for this! Done, and removed the pyapi module entirely.

[clarkzinzow]

LGTM!

[clarkzinzow]

Also, if the number of these uses increases, we can have a single attribute specification apply to an entire block of imports using this syntax:

Daft/src/daft-core/src/array/ops/cast.rs

Lines 27 to 40 in d032844

	#[cfg(feature = "python")]
	use {
	crate::array::pseudo_arrow::PseudoArrowArray,
	crate::datatypes::{ListArray, PythonArray},
	crate::ffi,
	crate::with_match_numeric_daft_types,
	log,
	ndarray::IntoDimension,
	num_traits::{NumCast, ToPrimitive},
	numpy::{PyArray3, PyReadonlyArrayDyn},
	pyo3::prelude::*,
	std::iter,
	std::ops::Deref,
	};

[clarkzinzow]

Actually, I think you were the one to make that change in your Duration dtype PR, I didn't know about that syntax until I saw your change! 😄