-
Notifications
You must be signed in to change notification settings - Fork 7
Feature/psf convolution refactor #218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
574d174
remove mask_shape from FFT
e3bbc5a
remove full_shape
ff26e25
readd full shape to fft_shape_from
109b009
use fft shape instrad of image.make.shape
5fa2cfb
move Kernel2D to operator Convoler, big refacotr incoming
a8b5b98
imports fixed
9852338
remove disable fft pad
0241480
refactor passes high level integration test
80331d3
Create ConvolverState to better manage its preloading
83bbd99
moved kernel_with_odd_dimensions_from to preprocess
c02f988
all of test_convolver now passes
30e20e0
fix fixtures
21cbd28
fix unit tests reffering to Kernel2D
35b09e0
fix more tests
0ff79d6
remove disable_fft_pad and imaigng plotter sends psf.kernel to plot
c3900fb
convolver imports exc
b6f12b1
all unit tests pass
9cfa1f3
fixc quirks in convolver state
45b53ef
reviews
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -9,7 +9,8 @@ | |||||
| ImagingSparseOperator, | ||||||
| ) | ||||||
| from autoarray.structures.arrays.uniform_2d import Array2D | ||||||
| from autoarray.structures.arrays.kernel_2d import Kernel2D | ||||||
| from autoarray.operators.convolver import ConvolverState | ||||||
| from autoarray.operators.convolver import Convolver | ||||||
| from autoarray.mask.mask_2d import Mask2D | ||||||
| from autoarray import type as ty | ||||||
|
|
||||||
|
|
@@ -26,11 +27,11 @@ def __init__( | |||||
| self, | ||||||
| data: Array2D, | ||||||
| noise_map: Optional[Array2D] = None, | ||||||
| psf: Optional[Kernel2D] = None, | ||||||
| psf: Optional[Convolver] = None, | ||||||
| psf_setup_state: bool = False, | ||||||
| noise_covariance_matrix: Optional[np.ndarray] = None, | ||||||
| over_sample_size_lp: Union[int, Array2D] = 4, | ||||||
| over_sample_size_pixelization: Union[int, Array2D] = 4, | ||||||
| disable_fft_pad: bool = True, | ||||||
| use_normalized_psf: Optional[bool] = True, | ||||||
| check_noise_map: bool = True, | ||||||
| sparse_operator: Optional[ImagingSparseOperator] = None, | ||||||
|
|
@@ -78,10 +79,6 @@ def __init__( | |||||
| over_sample_size_pixelization | ||||||
| How over sampling is performed for the grid which is associated with a pixelization, which is therefore | ||||||
| passed into the calculations performed in the `inversion` module. | ||||||
| disable_fft_pad | ||||||
| The FFT PSF convolution is optimal for a certain 2D FFT padding or trimming, which places the fewest zeros | ||||||
| around the image. If this is set to `True`, this optimal padding is not performed and the image is used | ||||||
| as-is. | ||||||
| use_normalized_psf | ||||||
| If `True`, the PSF kernel values are rescaled such that they sum to 1.0. This can be important for ensuring | ||||||
| the PSF kernel does not change the overall normalization of the image when it is convolved with it. | ||||||
|
|
@@ -93,50 +90,6 @@ def __init__( | |||||
| enable this linear algebra formalism for pixelized reconstructions. | ||||||
| """ | ||||||
|
|
||||||
| self.disable_fft_pad = disable_fft_pad | ||||||
|
|
||||||
| if psf is not None: | ||||||
|
|
||||||
| full_shape, fft_shape, mask_shape = psf.fft_shape_from(mask=data.mask) | ||||||
|
|
||||||
| if psf is not None and not disable_fft_pad and data.mask.shape != fft_shape: | ||||||
|
|
||||||
| # If using real-space convolution instead of FFT, enforce odd-odd shapes | ||||||
| if not psf.use_fft: | ||||||
| fft_shape = tuple(s + 1 if s % 2 == 0 else s for s in fft_shape) | ||||||
|
|
||||||
| logger.info( | ||||||
| f"Imaging data has been trimmed or padded for FFT convolution.\n" | ||||||
| f" - Original shape : {data.mask.shape}\n" | ||||||
| f" - FFT shape : {fft_shape}\n" | ||||||
| f"Padding ensures accurate PSF convolution in Fourier space. " | ||||||
| f"Set `disable_fft_pad=True` in Imaging object to turn off automatic padding." | ||||||
| ) | ||||||
|
|
||||||
| over_sample_size_lp = ( | ||||||
| over_sample_util.over_sample_size_convert_to_array_2d_from( | ||||||
| over_sample_size=over_sample_size_lp, mask=data.mask | ||||||
| ) | ||||||
| ) | ||||||
| over_sample_size_lp = over_sample_size_lp.resized_from( | ||||||
| new_shape=fft_shape, mask_pad_value=1 | ||||||
| ) | ||||||
|
|
||||||
| over_sample_size_pixelization = ( | ||||||
| over_sample_util.over_sample_size_convert_to_array_2d_from( | ||||||
| over_sample_size=over_sample_size_pixelization, mask=data.mask | ||||||
| ) | ||||||
| ) | ||||||
| over_sample_size_pixelization = over_sample_size_pixelization.resized_from( | ||||||
| new_shape=fft_shape, mask_pad_value=1 | ||||||
| ) | ||||||
|
|
||||||
| data = data.resized_from(new_shape=fft_shape, mask_pad_value=1) | ||||||
| if noise_map is not None: | ||||||
| noise_map = noise_map.resized_from( | ||||||
| new_shape=fft_shape, mask_pad_value=1 | ||||||
| ) | ||||||
|
|
||||||
| super().__init__( | ||||||
| data=data, | ||||||
| noise_map=noise_map, | ||||||
|
|
@@ -145,8 +98,6 @@ def __init__( | |||||
| over_sample_size_pixelization=over_sample_size_pixelization, | ||||||
| ) | ||||||
|
|
||||||
| self.use_normalized_psf = use_normalized_psf | ||||||
|
|
||||||
| if self.noise_map.native is not None and check_noise_map: | ||||||
| if ((self.noise_map.native <= 0.0) * np.invert(self.noise_map.mask)).any(): | ||||||
| zero_entries = np.argwhere(self.noise_map.native <= 0.0) | ||||||
|
|
@@ -163,36 +114,22 @@ def __init__( | |||||
|
|
||||||
| if psf is not None: | ||||||
|
|
||||||
| if not data.mask.is_all_false: | ||||||
| if use_normalized_psf: | ||||||
|
|
||||||
| image_mask = data.mask | ||||||
| blurring_mask = data.mask.derive_mask.blurring_from( | ||||||
| kernel_shape_native=psf.shape_native | ||||||
| psf.kernel._array = np.divide( | ||||||
| psf.kernel._array, np.sum(psf.kernel._array) | ||||||
| ) | ||||||
|
|
||||||
| else: | ||||||
| if psf_setup_state: | ||||||
|
|
||||||
| image_mask = None | ||||||
| blurring_mask = None | ||||||
| state = ConvolverState(kernel=psf.kernel, mask=self.data.mask) | ||||||
|
|
||||||
| psf = Kernel2D.no_mask( | ||||||
| values=psf.native._array, | ||||||
| pixel_scales=psf.pixel_scales, | ||||||
| normalize=use_normalized_psf, | ||||||
| image_mask=image_mask, | ||||||
| blurring_mask=blurring_mask, | ||||||
| mask_shape=mask_shape, | ||||||
| full_shape=full_shape, | ||||||
| fft_shape=fft_shape, | ||||||
| ) | ||||||
| psf = Convolver( | ||||||
| kernel=psf.kernel, state=state, normalize=use_normalized_psf | ||||||
|
||||||
| kernel=psf.kernel, state=state, normalize=use_normalized_psf | |
| kernel=psf.kernel, state=state, normalize=False |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,7 +4,7 @@ | |
|
|
||
| from autoarray.dataset.imaging.dataset import Imaging | ||
| from autoarray.structures.arrays.uniform_2d import Array2D | ||
| from autoarray.structures.arrays.kernel_2d import Kernel2D | ||
| from autoarray.operators.convolver import Convolver | ||
| from autoarray.mask.mask_2d import Mask2D | ||
|
|
||
| from autoarray import exc | ||
|
|
@@ -19,7 +19,7 @@ def __init__( | |
| exposure_time: float, | ||
| background_sky_level: float = 0.0, | ||
| subtract_background_sky: bool = True, | ||
| psf: Kernel2D = None, | ||
| psf: Convolver = None, | ||
| use_real_space_convolution: bool = True, | ||
| normalize_psf: bool = True, | ||
| add_poisson_noise_to_data: bool = True, | ||
|
|
@@ -95,7 +95,7 @@ def __init__( | |
| psf = psf.normalized | ||
|
||
| self.psf = psf | ||
| else: | ||
| self.psf = Kernel2D.no_blur(pixel_scales=1.0) | ||
| self.psf = Convolver.no_blur(pixel_scales=1.0) | ||
|
|
||
| self.use_real_space_convolution = use_real_space_convolution | ||
| self.exposure_time = exposure_time | ||
|
|
@@ -192,13 +192,12 @@ def via_image_from( | |
| psf=self.psf, | ||
| noise_map=noise_map, | ||
| check_noise_map=False, | ||
| disable_fft_pad=True, | ||
| ) | ||
|
|
||
| if over_sample_size is not None: | ||
|
|
||
| dataset = dataset.apply_over_sampling( | ||
| over_sample_size_lp=over_sample_size.native, disable_fft_pad=True | ||
| over_sample_size_lp=over_sample_size.native, | ||
| ) | ||
|
|
||
| return dataset | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Accessing
self.psf._state.blurring_maskdirectly assumes that_statehas already been initialized. However,_stateis None by default in the Convolver and is only set up when the Imaging constructor is called withpsf_setup_state=True(which only happens inapply_mask). When Imaging is created in other contexts (e.g.,from_fits, simulator),_statewill be None, causing an AttributeError. The code should callself.psf.state_from(self.mask)instead to ensure the state is created if it doesn't exist.