-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
possible bug in save_cog_with_dask
?
#185
Comments
Maybe assert should be assert self.final or can_flush(write) |
From what I can tell that's basically the same thing as what I did, right? Happy to make a PR for this if it would help. edit: would it make more sense to move the |
My concern is that I still don’t understand if the assert is overly aggressive or if there is an actual logic error, for example it could be triggered by certain rare Dask events, like work stealing behaviour of the scheduler. in general mpu code needs another pass of simplification, now that we know it works most of the time |
I hit the same issue today. Has there been any progress on this since October @Kirill888? @jerry-kobold do you find your fix to be stable in your company? Thanks ahead! |
At my company we have been using
save_cog_with_dask
very heavily to produce COGs from very large rasters. For certain raster/chunk combinations we have observed the function failing with anAssertionError
. I did some digging and isolated the problem to the following code:It seems that for certain combinations of raster dimensions and chunking scheme, a situation can arise in which
self.write_credits
is0
but the write has not completed. The following code below should reproduce the problem:The dimensions in this example were chosen because they match the dimensions of an actual raster for which this problem arose with this specific chunking. Using the time-honored method of
distributed.print
-based debugging, I isolated the error to the line that checks forself.write_credits < 1
. Playing around with the source, I noticed that changing this line tomakes the error go away and nothing else seems to break, but I don't know if that's the right decision or not.
The text was updated successfully, but these errors were encountered: