Open
Conversation
e6ef21f to
c74f443
Compare
c74f443 to
c3404bb
Compare
e7cb5a7 to
3a39c88
Compare
1eae893 to
4bd01c2
Compare
c32ac57 to
20f7741
Compare
47b055a to
dc38569
Compare
81bb9d2 to
b43ab00
Compare
Also, add a function to estimate a signal covariance from a timestream
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR implements two improvements for the lanczos regridder and adds new tasks to combine the regridding and stacking processes.
Lanczos updates
Signal variance estimate
Rather than using a single scalar as an estimate of the gridded signal variance, this PR adds the option to use an actual diagonal signal covariance derived from another dataset. Testing shows that using a proper covariance estimate improves interpolation around bright point sources while also reducing excess noise and artifacts introduced when using an excessively high variance on quieter parts of the sky. This PR:
signal_covargument in thesetupmethod oftransform.Regridderand, if provided, uses this container to make the inverse signal covariance for the interpolationtransform.Regridderandsidereal.SiderealRegriddercode a bitArtifact masking
Using a proper signal covariance seems to reduce/remove the bulk of regridding artifacts, but there are still some residual artifacts at the edge of heavily masked areas. From testing, this seems to occur when the main lobe of the regridding kernel is centred on flagged time samples. The estimate of the gridded sample comes only from the kernel side lobes, where small numerical errors end up being significant. This PR addresses this by projecting the data mask through the absolute value of the gridding kernel and masking samples below some fractional threshold.
Combined regridding and stacking
This is still somewhat experimental, and I have not yet tested whether the resulting stack is better using this method instead of just using the standard quarterstacking.
Having more time samples present in the Wiener filter regridder should in theory produce a better estimate of the underlying signal. Rather than gridding each individual day and taking a weighted average, as is done in the standard quarterstack pipeline, this PR adds tasks to split the gridding into steps which produce
The idea is to produce these two datasets for several days, sum them over all days, add an estimate of the gridded signal covariance, and then solve the deconvolution problem. This should have two advantages:
In order to make this process computationally feasible, the weights (noise) dataset, and thereby the noise covariance, is factorized into a frequency-time array and a frequency-baseline array. This PR does the folllwing:
regrid.band_wienercode to separate the gridding processcore.io.Truncateto get the truncation weight dataset from a class property, allowing that dataset to be produced on the fly