Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add alternative combining methods to rct.xarray.*_from_delayed #85

Open
bolliger32 opened this issue Aug 26, 2020 · 2 comments
Open

add alternative combining methods to rct.xarray.*_from_delayed #85

bolliger32 opened this issue Aug 26, 2020 · 2 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@bolliger32
Copy link
Collaborator

Paraphrased from @delgadom in #84:

It would be cool to combine our *_from_delayed rhg_compute_tools.xarray functions with the new xarray combine functions so you could combine based on coords or auto-combine. or just drop the dataarray_ and dataset_ from delayed functions and just provide dataarrays_ and datasets_ functions and point the users to these concat functions.

Workflow would just be:

futures = [ ... ] # flat list of dataarray futures with arbitrary non-overlapping coordinate relationships
da = xr.combine_by_coords(rhgx.dataarrays_from_delayed(futures))

futures = [[...], [...], ...] # nested list of datarrays with hierarchical structures
da = xr.combine_nested(rhgx.dataarrays_from_delayed(futures))

or even, if you want terrible performance and just don't care...

futures = [ ... ] # ordered flat list of dataarray futures with overlapping coordinate relationships
da = functools.reduce(lambda x, y: x.combine_first(y), rhgx.dataarrays_from_delayed(futures))

@bolliger32 bolliger32 added enhancement New feature or request good first issue Good for newcomers labels Aug 26, 2020
@delgadom
Copy link
Member

Yeah, or just get people to use native xarray and drop the concat functions. Turning futures into lists of dask arrays is already a huge help and then getting people to combine these with xr concat tools is probably the best?

@bolliger32
Copy link
Collaborator Author

yeah good point so maybe all that's needed is to remove the function that uses concat and add some hints/examples in the docstring of the functions that return a list of dask objects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants