pl.from_pandas with columns that mix strings and NAs results in TypeError

[luukhopman]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

> pl.from_pandas(pd.DataFrame({"col": ["a", pd.NA]}))

Log output

No response

Issue description

This issue is new in version 1.0.0.

Expected behavior

In polars==0.20.31 this works fine:

> pl.from_pandas(pd.DataFrame({"col": ["a", pd.NA]}))

test
str
"a"
null

I went through the breaking changes of polars==1.0.0 and this new error does not seem foreseen. Although it might be related to #16939.

Installed versions

--------Version info---------
Polars:               1.0.0
Index type:           UInt32
Platform:             Windows-10-10.0.22621-SP0
Python:               3.10.0 | packaged by conda-forge | (default, Nov 20 2021, 02:18:13) [MSC v.1916 64 bit (AMD64)]

----Optional dependencies----
adbc_driver_manager:  <not installed>
cloudpickle:          <not installed>
connectorx:           <not installed>
deltalake:            <not installed>
fastexcel:            <not installed>
fsspec:               2023.6.0
gevent:               <not installed>
great_tables:         <not installed>
hvplot:               <not installed>
matplotlib:           3.8.3
nest_asyncio:         1.6.0
numpy:                1.26.4
openpyxl:             3.1.2
pandas:               2.2.2
pyarrow:              15.0.0
pydantic:             <not installed>
pyiceberg:            <not installed>
sqlalchemy:           1.4.47
torch:                <not installed>
xlsx2csv:             0.8.2
xlsxwriter:           <not installed>

[stinodego]

Thanks for the report, can reproduce and confirmed this works on 0.20.31. Prioritizing this as 'high' as it's a 1.0 regression.

[stinodego]

Update:

The problem started with this PR, which I sort of figured: #16939

pd.NA is not a valid string value. It works if you pass None instead. But when parsing pandas inputs, we should accept pd.NA as null when nan_as_null is enabled (which it is by default).

[luukhopman]

Thanks for the fix 🐻‍❄️