BUG: Inconsistent behavior of Groupby with None values with filter (#… #63178
Conversation
rhshadrach
left a comment
rhshadrach
left a comment
There was a problem hiding this comment.
Thanks for the PR! Please always add tests. Does this also handle the tuple case on L667?
| for name in names | ||
| ) | ||
|
|
||
| elif any(isna(k) for k in self.indices.keys()): |
There was a problem hiding this comment.
This check is expensive - this function is only ever called currently with names a list of length 1, and the rest of the method is O(1) in terms of self.indices. It's called from the inner loop of DataFramGroupBy.fitler as we're iterating over each group. This seems avoidable.
I believe we could change this function to just accept a single name (rather than a list) and then have a special case:
if isna(name):
return self.indices.get(np.nan, [])| names = (converter(name) for name in names) | ||
|
|
||
| return [self.indices.get(name, []) for name in names] | ||
| indices = {np.nan if isna(k) else k: v for k, v in self.indices.items()} |
There was a problem hiding this comment.
It seems better to do this on indices cached property directly, and only in the case where there is a NaN value with if not self.dropna and self.result_index.hasnans.
2 participants
…62501)
doc/source/whatsnew/v2.3.4.rstfile if fixing a bug or adding a new feature.