comm_subplan_elim segfault on main StackOverflow

[cmdlineluser]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl

lf = pl.LazyFrame({"A": []})

lf = pl.concat([lf] * 8192)

lf.with_columns(1).collect(comm_subplan_elim=True) # default, but works with False 
# zsh: segmentation fault

Log output

found multiple sources; run comm_subplan_elim

Issue description

I guess this is related to the new CSE work.

It only seems to trigger for >= 8192

Expected behavior

No segfault.

Installed versions

main

[mcrumiller]

My segfaults do start at lf = pl.concat([lf] * 12289) however. Any thing less succeeds.

import polars as pl
lf = pl.LazyFrame({"A": []})
lf = pl.concat([lf] * 12888)
lf.with_columns(1).collect(comm_subplan_elim=True)
# shape: (0, 2)
# ┌──────┬─────────┐
# │ A    ┆ literal │
# │ ---  ┆ ---     │
# │ null ┆ i32     │
# ╞══════╪═════════╡
# └──────┴─────────┘

lf = pl.LazyFrame({"A": []})
lf = pl.concat([lf] * 12889)
lf.with_columns(1).collect(comm_subplan_elim=True)
# Segmentation fault

[ritchie46]

This is a stack overflow. We hope to be able fix this properly by being able to set the stack on the heap if it grows too big, but it isn't easy.

[ritchie46]

mostly fixed by #15355