-
-
Notifications
You must be signed in to change notification settings - Fork 18.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
API: ignore empty range/object dtype in Index setop operations (string dtype compat) #60797
base: main
Are you sure you want to change the base?
API: ignore empty range/object dtype in Index setop operations (string dtype compat) #60797
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm - I think this is a pretty reasonable workaround for the time being
Agreed. While it's undesirable to me to ignore empty object dtype when using e.g. concat, using StringDtype should make having object dtype in the first place much more of an edge case. And the impact of this on e.g. adding columns to an empty DataFrame is worth the tradeoff. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm, needs a whatsnew, yea? I'd suggest a section at the top, I expect this to be somewhat commonly hit.
Exploring one option to address #60338
This makes that inserting/union etc with an empty RangeIndex or empty Index with object dtype (the types of index that get created for empty objects by default), ignores the dtype of that empty index.
This improves the workflow of creating an empty dataframe and adding columns or resetting the index (you can see the removed xfails). Also most of the test changes are changes that are essentially reverting previous edits (typically adding an explicit
dtype=object
) to get the tests passing for infer_string, and I think almost all those test changes show a better behaviour now.I still want to add some explicit tests for the original use case (although everything is implicitly already covered in other tests, I think)