Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ENH: Support skipna parameter in GroupBy min, max, prod, median, var, std and sem methods #60752

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

ENH: Support skipna parameter in GroupBy min, max, prod, median, var, std and sem methods #60752

wants to merge 12 commits into from
+405 −51

Conversation

snitish
Copy link
Contributor

@snitish snitish commented Jan 22, 2025 •
edited
Loading

Second (and final) batch of GroupBy reductions being enhanced to support the skipna parameter.

@snitish snitish marked this pull request as draft January 22, 2025 01:33
WillAyd
WillAyd requested changes Jan 22, 2025
View reviewed changes
pandas/_libs/groupby.pyx Outdated Show resolved Hide resolved
@snitish snitish changed the title ENH: Support skipna parameter in GroupBy prod, var, std and sem methods ENH: Support skipna parameter in GroupBy min, max, prod, var, std and sem methods Jan 22, 2025
@snitish snitish marked this pull request as ready for review January 22, 2025 04:14
@snitish snitish changed the title ENH: Support skipna parameter in GroupBy min, max, prod, var, std and sem methods ENH: Support skipna parameter in GroupBy min, max, prod, median, var, std and sem methods Jan 22, 2025
WillAyd
WillAyd reviewed Jan 22, 2025
View reviewed changes
Copy link
Member

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good. One question

pandas/core/_numba/kernels/min_max_.py Outdated Show resolved Hide resolved
WillAyd
WillAyd requested changes Jan 22, 2025
View reviewed changes
pandas/_libs/groupby.pyx Outdated Show resolved Hide resolved
WillAyd
WillAyd reviewed Jan 22, 2025
View reviewed changes
Copy link
Member

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few smaller comments. @rhshadrach mind taking a look as well?

pandas/_libs/groupby.pyx Show resolved Hide resolved
pandas/core/resample.py Show resolved Hide resolved
pandas/core/resample.py Show resolved Hide resolved
@rhshadrach rhshadrach added Enhancement Groupby Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate labels Jan 25, 2025
@rhshadrach rhshadrach added this to the 3.0 milestone Jan 25, 2025
rhshadrach
rhshadrach requested changes Jan 25, 2025
View reviewed changes
Copy link
Member

@rhshadrach rhshadrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Can you add a test (adding to your current parametrizations would be fine) where the entire group is NA.

pandas/_libs/groupby.pyx Show resolved Hide resolved
pandas/core/_numba/kernels/var_.py Outdated Show resolved Hide resolved
pandas/core/groupby/groupby.py Show resolved Hide resolved
pandas/core/resample.py Show resolved Hide resolved
pandas/core/resample.py Show resolved Hide resolved
@snitish
Copy link
Contributor Author

snitish commented Jan 25, 2025

Looks great! Can you add a test (adding to your current parametrizations would be fine) where the entire group is NA.

Thanks for the review @rhshadrach. Added the all-NA tests and responded to comments.

@rhshadrach
Copy link
Member

Failure on the future infer string is unrelated (and is fixed by #60796). Rerunning Ubuntu 310 just to be sure.

@snitish snitish requested a review from rhshadrach January 31, 2025 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@WillAyd WillAyd WillAyd requested changes

@rhshadrach rhshadrach Awaiting requested review from rhshadrach rhshadrach is a code owner

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
Enhancement Groupby Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
Projects
None yet
Milestone
3.0
Development

Successfully merging this pull request may close these issues.

ENH: enable skipna on groupby reduction ops
3 participants
@snitish @rhshadrach @WillAyd