Re-work API for list.to_struct()

[nameexhaustion]

Description

This is what the current API looks like:

def to_struct(
        self,
        n_field_strategy: Literal["first_non_null", "max_width"] = "first_non_null",
        fields: Sequence[str] | Callable[[int], str] | None = None,
        upper_bound: int = 0,
    ) -> Expr:

This is problematic for Lazy execution, as if a user calls list.to_struct() without specifying the field names, it can lead to confusing errors in subsequent expressions as we cannot determine the schema during IR resolution - e.g.

• list.to_struct().struct.unnest() fails #19812
• struct.prefix_fields() fails after list.to_struct() with InvalidOperationError #20137
• Some struct functions do not work on .list.to_struct #16370
• .list.to_struct() has non-deterministic behavior #16450

Specifically, the error message could look something like struct.prefix_fields() not supported for dtype Unknown. It's not immediately clear that this is due to having done a list.to_struct() without specifying field names earlier in the query.

I would propose something like:

def to_struct(
    self,
    field_names: Sequence[str] | Callable[[int], str],
    *,
    infer_fields: Literal["first_non_null", "max_width"] | None = None,
) -> Expr:

Where we would forbid arguments that have an ambiguous number of output columns when executing on a LazyFrame:

Parameters	Series	DataFrame/LazyFrame
list.to_struct(['a', 'b'])	✅	✅
list.to_struct(field_names=lambda x: f"c{x}")	error: infer_fields must be specified
list.to_struct(field_names=lambda x: f"c{x}", infer_fields="first_non_null")	✅	❌
list.to_struct(field_names=lambda x: f"c{x}", infer_fields="max_width")	✅	❌

Notes

• We should probably also update arr.to_struct