astroquery
astroquery copied to clipboard
astropy
AstrometryNet providing different sky positions for similar frames.
I have been using astroquery.astrometry_net for performing astrometry on the science frames. However, today I came across a problem: I have two similar frames in the B band, and I tried performing astrometry on both the frames. However, the sky positions returned in both images are totally different. I wonder what could be the reason behind it. I am attaching screenshots for both the frames with updated WCS coordinates. (Please note the different $\alpha$ and $\delta$ values in the DS9 window.)
Can anyone please let me know if you have faced this before or am I missing something regarding this?
This looks to be an astrometry.net issue, please try to use their web interface and if you see the same issue report it to them directly as we only provide the interface to interact with the website, but not the algorithm or computing.
@bsipocz Thanks for the response. Actually, I just tried the astrometry.net web interface for the same frame. Surprisingly, it runs just fine in this case! I still don't know what caused the problem with this particular frame.
Ohh, interesting and very puzzling.
Could you try to clean the cache, and retry again? If it still produces the issue we may ask @mwcraig to see whether he has any ideas for smoking out the bug or even know a fix.
@viveikjha could you provide more specifics? i.e., what solutions are obtained, and what's the right solution? I'm not sure that will lead us to a clear answer, but that's the minimum information we'll need. If there's a genuine problem in astroquery that's not present in astrometry.net, the only explanation that comes to mind is a mismatch of solution indices - but it seems strange to have that happen as a one-off case.
@keflavich Actually I was not able to attach fits files, so I am attaching them as a google drive link here. The problems are:
- In frame b1.fits, the obtained coordinates after astrometry are totally different from the ones that we expect.
- In frame b2.fits, the obtained coordinates are as they should be.
As @bsipocz suggested, I uploaded the same image to astrometry.net and obtained the correct solution.
link to the fits files: https://drive.google.com/file/d/1zVFdvvp9NBftYQE7wAtYOelynValwTKm/view?usp=sharing https://drive.google.com/file/d/1KQYoSUmdeqap__Fb6r4OXA4Bal92C1B-/view?usp=sharing
Took a very quick look at this tonight (finally!) and the solutions based on the source lists are indeed different. My best guess is that it has something to do with the source extraction. In the b2 image the photutils source detection correctly identifies the very bright star in the image as the brightest source. In the b1 image that is NOT identified as the brightest source.
Not sure what the root cause is or how to fix it, to be honest...
What about trying to exclude pixels above a certain threshold from the source detection?
I think we really need that star to be present as a bright source for the field to match. Another approach would be to downsample, like astrometry.net itself does, get the source positions, upsample them back to the original size and feed those positions to astrometry.net.
I'm not going to have time to investigate that any time soon though....
Why do we need to do any of that? I mean the purpose here should be to upload the image to astrometry.net, just as one would do on the webform. Then the results shouldn't be different, or is there API behaves differently?
Right now there are three ways to get a solution from the astroquery astrometry.net module:
- Upload a list of sources you have generated your self.
- Provide an image, run source detection using photutils
- Provide an image, upload the image to astrometry.net
If you provide an image and you have photutils installed then 2 above is the default, primarily because it means moving much less data.
Option 3 is used if there is no photutils or a particular keyword is set (might be force_upload?)
Ok, but then any discrepancies with method 2 is not an astroquery problem. And in fact I don't think we should deal with any of that, but only provide a way to upload the catalog, and whatever way the user produces it is beyond our remit.
I've always been surprised by the default behavior changing whether or not photutils is installed - I wonder if it would be better to switch to a mode where using photutils internally is optional and opt-in?
Hi everyone...I'm glad this topic is alive again. Just to confirm, I didn't have photutils installed on my system at that time. Currently, I am using the locally downloaded index files for plate solving, as I have a larger number of fits files so can't confirm whether this issue with astroquery persists or not.
I've always been surprised by the default behavior changing whether or not photutils is installed - I wonder if it would be better to switch to a mode where using photutils internally is optional and opt-in?
That is a huge, huge oversight from the review side. Photutils is not declared as an optional dependency, and we have decided a number of times already that we don't do any opinionated (not the right word, but running source detection certainly has a wide range of sensible configurations) extra functionality to pre- or post-process data before remote access.
So, I would strongly suggest to remove any photutils functionality, and instead point to external resources of how to generate an input catalogue for an astrometry.net query.
cc @keflavich
@bsipocz We do have some utility & convenience functions in other tools, but we host them outside of the core.py file and don't maintain them as rigorously.
I agree with the general sentiment, though: it would make sense to remove the photutils convenience tool from solve_from_image and put it into its own code (and/or, add it as a documentation example).
A more tool-agnostic approach, if we want to enable local source finding, would be appropriate: let solve_from_image take some f(data) that returns an X,Y source list and pass that to solve_from_source_list. Then the photutils example can be provided as a reference implementation.
We do have some utility & convenience functions in other tools, but we host them outside of the core.py file and don't maintain them as rigorously.
we have removed many of those recently, I'm thinking about finding chart builders, and other broken bits an pieces.
As for astrometry.net, I think there are two approaches that should be supported. 1) Query the API with a source list that the user provides. We can link to photoutils docs, etc, but we don't do any of it here. 2) Query the API with uploading the image.
Just to confirm, I didn't have photutils installed on my system at that time.
In that case, I think this issue can be closed. I have no idea what the problem was at the time, but when I force image upload now both images lead to the same (correct) WCS.
As for astrometry.net, I think there are two approaches that should be supported. 1) Query the API with a source list that the user provides. We can link to photoutils docs, etc, but we don't do any of it here. 2) Query the API with uploading the image.
Could we make a new issue for this? I can review a PR between now and early september but don't anticipate having the time to write it, and will need some guidance as to whether you want a deprecation period or not.