Add Reproject plugin for reference viewer

[ejeschke]

• This is a plugin that basically provides a GUI for doing WCS reprojections using the AstroPy "reproject" module.
• Provided mostly as a basis for further discussion on How does Ginga handle image distortion? #328

[ejeschke]

This is mostly just to advance the discussion in #328. I think probably the support needs to be baked in a little deeper than just in a plugin, but this is a strawman to get ideas flowing.

To test this you need to have the AstroPy reproject module installed.

To use: start the Reproject plugin from the Operations->Utils menu (or Plugins->Utils). Load an image into the channel that you want to use as the reprojecting WCS (e.g. by drag drop). Click "Set WCS". Now load any image containing data that you want to reproject and click "Reproject". The image will be reprojected and added to the channel. You can manipulate the reprojection parameters using the GUI.

Caveats:

• the reprojected image may be clipped. I cannot see a way around this using the current reproject module (even though it should be quite possible)
• the reprojected image is currently shown with NaNs replacing unused pixels
• I have not updated any documentation yet in this PR

[ejeschke]

ping @pllim, @eteq, @hcferguson, @larrybradley

[ejeschke]

Added a WCS to the reprojected image.

[ejeschke]

Screenshot, using 2MASS example in reproject documentation:

[pllim]

@ejeschke , two things:

1. I have one big image uncorrected for distortion, I want to un-distort it for display but only the displayed portion of it. This plugin does not seem to handle that use case? If it does, I failed to figure it out from the docstring.
2. Having the entire big image (2k by 4k) displayed and clicking the reproject button, the viewer got stuck while the reprojection algorithm churns away. If we want to go this route with using plugin, we probably need some sort of progress bar display like the Mosaic plugin. Or simply refuse to process a too-big array.

I also encountered segfault at some point but I don't know how to reproduce it.

[ejeschke]

Rebased.

[ejeschke]

I have one big image uncorrected for distortion, I want to un-distort it for display but only the displayed portion of it. This plugin does not seem to handle that use case? If it does, I failed to figure it out from the docstring.

Well, that's what I mean when I said (above):

probably the support needs to be baked in a little deeper than just in a plugin, but this is a strawman to get ideas flowing

Ideally, the distortion should be corrected at image load time. That way, you only pay for it once, especially if it is compute intensive.

But, no, this plugin does not do just the displayed portion. We might be able to refactor it to work on only the displayed area, I would think.

[ejeschke]

Having the entire big image (2k by 4k) displayed and clicking the reproject button, the viewer got stuck while the reprojection algorithm churns away. If we want to go this route with using plugin, we probably need some sort of progress bar display like the Mosaic plugin. Or simply refuse to process a too-big array.

I thought that too, but I'm not sure offhand how to monitor the reproject call so that it would report its progress.

[ejeschke]

We could run the reproject call in another thread--that would be easy. The viewer would still be responsive. I'll see if I can push up a fix for that.

[ejeschke]

We could run the reproject call in another thread--that would be easy. The viewer would still be responsive. I'll see if I can push up a fix for that.

@pllim, if you try the plugin now you will see that it remains responsive during the reprojection. I also added a little status message in the plugin.

[ejeschke]

@pllim, does drizzle do what you want for STScI images? Has any put together a drizzle plugin for ginga?

[ejeschke]

Also, can I get a copy of your large image and also your "corrected one", to test with this? You could put them in the Box drive, if you folks are still using that...

[hcferguson]

@pllim @ejeschke There was an effort started some time ago to put drizzle into reproject. That's going to get some attention again over the next few months by Mihai Cara and others.

[pllim]

TBH, I still think it is a bad idea to have a GUI do the drizzling/reprojection on-the-fly for the whole image. It is computationally intensive and the image can be pre-processed outside the GUI anyway. I just don't see the point.

I think @hcferguson said one thing that can be useful is to use this as a way to explore which are the optimal parameters to use for the algorithm, but then the parameters that are exposed depend on the use case. So, I don't see much value in that either but maybe Harry can clarify if I misunderstood.

[hcferguson]

The main use case is for viewing multiple images with the same pixel scale and rotation in different windows. Generally one can ignore higher-order distortions for doing that. And for speed, one could do these all on just the portion viewed. Also one could in principle do this only for the bit depth viewed (i.e. on 16 or 32 bit integer data rather than floats). And there are in principle methods to use GPUs for such transformations, although I don't know how portable they are.

I suppose one relatively common use-case that stresses doing the whole image would be to overlay a mask (e.g. cosmic rays) from an unrectified image onto the rectified image. There it's necessary to do the full reprojection, and one generally wants a method that does some nearest-neighbor interpolation so the reprojected mask is still binary.

That is to say ... I can see the benefit of having lightning fast reprojections that take shortcuts, as well as a full reprojection...

[pllim]

Thanks for the clarification, @hcferguson . Can you please upload some test images (maybe 3 would be sufficient?) to https://stsci.app.box.com/folder/3381606676 for @ejeschke ?

[ejeschke]

Rebased.

[ejeschke]

the reprojected image may be clipped. I cannot see a way around this using the current reproject module (even though it should be quite possible)

I fthink I've figured this out. With the latest commit and example images it now can get the fully rotated result.

[ejeschke]

The main use case is for viewing multiple images with the same pixel scale and rotation in different windows. Generally one can ignore higher-order distortions for doing that. And for speed, one could do these all on just the portion viewed.

Isn't that covered pretty well with the WCSMatch plugin?

[ejeschke]

There is another candidate PR (#816) that I think works a little more closely to the workflow @pllim was describing. You don't need two WCSes and it doesn't use reproject, instead it tries to remap the pixels using the SIP WCS in the image.

I'm still a little unclear if this is the kind of "undistortion" being described in this issue, but it seems to do something akin to barrel distortion type adjustment to a couple of the HST images I tried.

[pllim]

p.s. This needs a rebase to fix the ginga command line.

[pllim]

(say, the one at 11:15:58.73, 1:30:51.12)

For my own future reference, this is the one at X=748 Y=479 in the DRZ image.

[pllim]

Using the three files that @hcferguson provded in #813 (comment) ...

1. WCS match after load without any other manipulations. DRZ is the reference frame. DRZ is on upper left.

2. I use DRZ to set WCS, then reprojected one of the FLCs with the "Reproject" button. Then I move the reprojected result to another channel so I can use WCSMatch. DRZ and reproject result do not align at all. DRZ is on the left.

3. Still using the Reproject plugin in Step 2 above (probably still has DRZ as reference WCS from clicking that "Set WCS" above), I now press the "Undistort" button for one of the FLC. Then again, I move the undistorted FLC to another channel for WCSMatch to work. Much better but if you look carefully, the alignments are not perfect (i.e., things shift a little when you blink them). DRZ is on the left.

[ejeschke]

Rebased.

[ejeschke]

Rebased