Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to handle NaN and inf values and non-convergent models (logp < -300) in the counterfactual output #93

Open
1 task done
Tracked by #91
A-Buch opened this issue Jul 24, 2023 · 0 comments
Open
1 task done
Tracked by #91

Comments

@A-Buch
Copy link

A-Buch commented Jul 24, 2023

FIXME in rechunk_necdf() there should not be NaN values in the counterfactual if the factual data (input file) is not NaN for this cell. I am not sure what happens if the factual data is nan only for some of the time steps. That should not happen because it is a reanalysis dataset.
If NaN values are in the counterfactual but not the factual are there anyways, it should be further investigated and discussed how to handle those cases. If they can't be avoided, should they be set to the factual value (as currently) or something else?

Todo in attrici.postprocess.py

  • In replace_nan_inf_with_orig The part to count NaN values does not work correctly yet. What should be done is to count only cells in cfactual which are nan but not in factual something like:
    isnan = np.isnan(v) & (~np.isnan(v_orig))
    This code snipped, currently doesn't work because v and v_orig are masked arrays. Should change how v and v_orig are loaded from the netcdf file (e.g. with xarray), such that they are normal arrays. Note that this masking has nothing to do with the landseamask
@A-Buch A-Buch changed the title FIXME large nc file even that it contains only many nan values, check why it is that large and fix it e.g. adapt deflate_level, make as a new git issue FIXME in rechunk_necdf() how nan values are stored Jul 24, 2023
@SimonTreu SimonTreu changed the title FIXME in rechunk_necdf() how nan values are stored How to handle NaN and inf values and non-convergent models (logp < -300) in the counterfactual output Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant