Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix error when saving array with NaNs (occurs with corr_img when use_corr_img is True) #1385

Merged
merged 1 commit into from
Aug 9, 2024

Conversation

ethanbb
Copy link
Contributor

@ethanbb ethanbb commented Aug 9, 2024

Description

The function recursively_save_dict_contents_to_group checks that an ndarray was written correctly to the HDF5 file by calling np.array_equal, but it currently does not set the option equal_nan=True, meaning that if there are any NaNs in the array, it returns false. Regardless of whether NaNs are expected, this is probably not what we want if we are just doing a fidelity check.

I initially tried just changing this to true, but apparently it can fail with a TypeError ("ufunc 'isnan' not supported for the input types") in some cases. So I changed it to only be true if the ndarray is of a floating-point type, which seems to work.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

Has your PR been tested?

Have not tested beyond this; may be good to add a unit test for it, which I can come back and do later.

@pgunn pgunn merged commit 808b22f into flatironinstitute:dev Aug 9, 2024
3 checks passed
@pgunn
Copy link
Member

pgunn commented Aug 9, 2024

Looks great, thanks!

@ethanbb
Copy link
Contributor Author

ethanbb commented Aug 9, 2024

Sorry, this has a typo. It should be item.dtype.kind == 'f'. But now I'm confused why it fixed the error I was getting, so looking into that...

@pgunn
Copy link
Member

pgunn commented Aug 9, 2024

Ah, I may have merged too soon. If you want to followup with another PR..

@ethanbb
Copy link
Contributor Author

ethanbb commented Aug 9, 2024

OK got it, so dtype('float32') compares equal with 'f', which is why it worked for me, but other floating-point dtypes do not. So the right thing to do is to use the kind.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants