PanicException for struct append when using POLARS_MAX_THREADS=1

[coastalwhite]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl
import os

os.environ['POLARS_MAX_THREADS'] = '1'

df = pl.DataFrame(
    {
        "nested": [
            [{"a": 1}],
            [],
            [{"a": 3}],
        ]
    }
)
assert pl.concat([df[0], df[1], df[2]]).select(
    pl.col("nested").list.to_struct()
).to_dict(as_series=False) == {
    "nested": [{"field_0": {"a": 1}}, {"field_0": None}, {"field_0": {"a": 3}}]
}

Log output

run ProjectionExec
thread '<unnamed>' panicked at crates/polars-core/src/frame/mod.rs:1105:36:
should not fail: SchemaMismatch(ErrString("cannot append series, data types don't match"))
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
  File "/home/johndoe/Projects/polars/t5.py", line 12, in <module>
    assert pl.concat([df[0], df[1], df[2]]).select(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/johndoe/Projects/polars/py-polars/polars/dataframe/frame.py", line 9021, in select
    return self.lazy().select(*exprs, **named_exprs).collect(_eager=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/johndoe/Projects/polars/py-polars/polars/lazyframe/frame.py", line 2055, in collect
    return wrap_df(ldf.collect(callback))
                   ^^^^^^^^^^^^^^^^^^^^^
pyo3_runtime.PanicException: should not fail: SchemaMismatch(ErrString("cannot append series, data types don't match"))

Issue description

Panic should not be happening, it seems this was introduced by #19218.

Expected behavior

No panic

Installed versions

1.12.0