BUG: ValueError when calling groupby + fillna on StringDType column

[pspeter]

Versions

pd.show_versions()

 INSTALLED VERSIONS
------------------
commit           : f2c8480af2f25efdbd803218b9d87980f416563e
python           : 3.8.7.final.0
python-bits      : 64
OS               : Windows
OS-release       : 10
Version          : 10.0.18362
machine          : AMD64
processor        : Intel64 Family 6 Model 142 Stepping 12, GenuineIntel
byteorder        : little
LC_ALL           : None
LANG             : None
LOCALE           : English
pandas           : 1.2.3
numpy            : 1.19.5
pytz             : 2021.1
dateutil         : 2.8.1
pip              : 21.0.1
setuptools       : 49.2.1
Cython           : None
pytest           : 6.2.2
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : 2.11.3
IPython          : 7.21.0
pandas_datareader: None
bs4              : None
bottleneck       : None
fsspec           : None
fastparquet      : None
gcsfs            : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : 3.0.0
pyxlsb           : None
s3fs             : None
scipy            : 1.6.1
sqlalchemy       : None
tables           : None
tabulate         : None
xarray           : None
xlrd             : None
xlwt             : None
numba            : None

Description

When doing a groupby and then a fillna on a column of dtype StringDType, pandas throws a ValueError: StringArray requires a sequence of strings or pandas.NA. This does not seem to happen with other (extension) dtypes afaik.
The error only occurs if there are still NAs left after the fillna. It does not occur without the groupby.

Example

>> pd.DataFrame({"a": pd.array([None, "a"], dtype="string"), "b": [0, 0]}).groupby("b").ffill()

Traceback (most recent call last):
  File "C:\venv\lib\site-packages\IPython\core\interactiveshell.py", line 3437, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-22-a665c967b563>", line 1, in <module>
    pd.DataFrame({"a": pd.array([None, "a"], dtype="string"), "b": [0, 0]}).groupby("b").ffill()
  File "C:\venv\lib\site-packages\pandas\core\groupby\groupby.py", line 1953, in pad
    return self._fill("ffill", limit=limit)
  File "C:\venv\lib\site-packages\pandas\core\groupby\groupby.py", line 1919, in _fill
    return self._get_cythonized_result(
  File "C:\venv\lib\site-packages\pandas\core\groupby\groupby.py", line 2673, in _get_cythonized_result
    result = algorithms.take_nd(values, result)
  File "C:\venv\lib\site-packages\pandas\core\algorithms.py", line 1699, in take_nd
    return arr.take(indexer, fill_value=fill_value, allow_fill=allow_fill)
  File "C:\venv\lib\site-packages\pandas\core\arrays\_mixins.py", line 78, in take
    return self._from_backing_data(new_data)
  File "C:\venv\lib\site-packages\pandas\core\arrays\numpy_.py", line 190, in _from_backing_data
    return type(self)(arr)
  File "C:\venv\lib\site-packages\pandas\core\arrays\string_.py", line 195, in __init__
    self._validate()
  File "C:\venv\lib\site-packages\pandas\core\arrays\string_.py", line 200, in _validate
    raise ValueError("StringArray requires a sequence of strings or pandas.NA")
ValueError: StringArray requires a sequence of strings or pandas.NA

>> pd.DataFrame({"a": pd.array(["a", None], dtype="string"), "b": [0, 0]}).groupby("b").ffill()
Out[31]: 
   a
0  a
1  a

Edit: Updated pandas, still an issue in 1.2.3

[pspeter]

Sorry for not using the issue template, it wasn't working earlier when I reported this

[mzeitlin11]

Thanks for the report @pspeter! This is currently working in master, looks to have been fixed by #39446. Since that wasn't designed to fix this, could use a test.

[shiv-io]

take

[shiv-io]

@mzeitlin11, I understand the issue is fixed in master as you mentioned but I'd like to work on adding a test for this. I'm a first-time contributor to pandas, and would appreciate some direction.

I've gone through the contributing guide and know my way around git, etc.

What's the best way to assess which module this test should go into? My best guess is pandas/tests/frame/methods/test_fillna.py, since it is the fillna() method that causes the error, but I could be wrong.

[mzeitlin11]

@mzeitlin11, I understand the issue is fixed in master as you mentioned but I'd like to work on adding a test for this. I'm a first-time contributor to pandas, and would appreciate some direction.

Awesome, welcome to pandas!

What's the best way to assess which module this test should go into? My best guess is pandas/tests/frame/methods/test_fillna.py, since it is the fillna() method that causes the error, but I could be wrong.

I think the groupby takes precedence here, so it should go in pandas/tests/groupby/test_missing.py (looks like there are a few similar tests in that file).