-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Could we defer to flox for GroupBy.first
?
#9647
Comments
Yes , the minor complication is that we should dispatch Closing #8025 in favor of this one. Out of curiosity how many groups does your problem have? |
Sorry I missed #8025, I thought I searched; I guess
About 15K... |
Do you end up using dask for this, or just numbagg? Are these groups randomly distributed along the dimension, or are there patterns to how they are distributed (e.g. are they sequential)? Just curious... |
I ended up just leaving it running for hours!
Yes they're largely sequential! |
1. Use flox where possible. 2. Use simple indexing where possible. Closes pydata#9647
This reverts commit a848044. Opens pydata#9647 Closes pydata#9993
Is your feature request related to a problem?
I was wondering why a
groupby("foo").first()
call was going so slowly — I think we run a python loop for this, rather than calling into flox:xarray/xarray/core/groupby.py
Lines 1218 to 1231 in b9780e7
Describe the solution you'd like
Could we call into flox? Numbagg has the routines...
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: