Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix arrow groupby na #60777

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

asharmalik19
Copy link

@asharmalik19 asharmalik19 force-pushed the fix-arrow-groupby-na branch 3 times, most recently from ac8116b to 0e61611 Compare January 27, 2025 22:00
@rhshadrach rhshadrach self-requested a review January 30, 2025 21:41
Comment on lines +1212 to +1213
# dictionary encode does nothing if an already encoded array is given
data = data.cast(data.type.value_type)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're explaining why you have the cast here, but I'm not quite understanding it. Can you give a bit more details (just as a response here, and we can decide if the comment should be updated later).

Comment on lines +618 to +619
- Fixed bug in :meth:`ArrowExtensionArray.factorize` where NA values were dropped when input was dictionary-encoded even when dropna was set to False(:issue:`60567`)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move this to an appropriate section below, I think ExtensionArray.

@@ -3330,6 +3330,19 @@ def test_factorize_chunked_dictionary():
tm.assert_index_equal(res_uniques, exp_uniques)


def test_factorize_dictionary_with_na():
# Test that factorize properly handles NA values in dictionary arrays
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you start the test with a comment referencing the issue, e.g.

# GH#60567

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: groupby with dropna=False and pa.dictionary drops NA values
2 participants