Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WFAU module has no way of identifying deprecated WFCAM images #2808

Open
virajkaram opened this issue Aug 12, 2023 · 4 comments · May be fixed by #2809
Open

WFAU module has no way of identifying deprecated WFCAM images #2808

virajkaram opened this issue Aug 12, 2023 · 4 comments · May be fixed by #2809
Labels
wfau

Comments

@virajkaram
Copy link

The WFCAM Science Archive has the concept of deprecated WFCAM images (see e.g. the description of deprecated column here http://wsa.roe.ac.uk/www/gloss_d.html). However the WFAU module doesn't seem to have the option of only downloading undeprecated images?
Here is my best guess for where the deprecated value might be specified in the code, but the query always returns both deprecated and undeprecated images.

I realize this functionality may be missing from the WFCAM archive itself, as the web-form does not have a deprecated field. Upon submitting the web-form, a table is returned that has the deprecated values for each image, and the user can manually choose to only download undeprecated images. I was wondering if anyone here has any insights on how to do this programatically through astroquery?
@keflavich keflavich added the wfau label Aug 12, 2023
@keflavich
Copy link
Contributor

@virajkaram since the web form doesn't support it, we can't directly enable getting only non-deprecated images, I believe. If there is a hidden parameter (dep, as you indicated), you could experiment with changing that and see if it has the desired effect, but it looks like we're already specifying dep=0.

Does get_image_list return any metadata about the deprecation status? If so, we could parse that in astroquery.

It would help if you could provide a minimal or at least a small example in which you indicate which files are and are not deprecated; we can perhaps then suss out how to get only the non-deprecated files.

@virajkaram
Copy link
Author 

from astropy.coordinates import SkyCoord
import astropy.units as u
from astroquery.ukidss import UkidssClass

crd = SkyCoord(ra=211.3194905, dec=54.413845, unit=(u.deg,u.deg))
q = UkidssClass()
q.database = 'UHSDR2'
q.get_image_list(crd, waveband='all')

returns

['http://wsa.roe.ac.uk/cgi-bin/getFImage.cgi?file=/disk44/wsa/ingest/fits/20140520_v5/w20140520_01594_st.fit&mfid=7427416&extNo=4&lx=1279&hx=1429&ly=195&hy=345&rf=0&flip=1&uniq=5034_318_6_85646_1&xpos=75.5&ypos=76.2&band=J&ra=211.3194905&dec=54.413845',
 'http://wsa.roe.ac.uk/cgi-bin/getFImage.cgi?file=/disk52/wsa/ingest/fits/20150112_v5/w20150112_02350_st.fit&mfid=8351228&extNo=4&lx=1275&hx=1425&ly=194&hy=344&rf=0&flip=1&uniq=5034_318_6_85646_2&xpos=76.2&ypos=76.4&band=J&ra=211.3194905&dec=54.413845',
 'http://wsa.roe.ac.uk/cgi-bin/getFImage.cgi?file=/disk53/wsa/ingest/fits/20150125_v5/w20150125_02715_st.fit&mfid=8327025&extNo=4&lx=1275&hx=1425&ly=195&hy=345&rf=0&flip=1&uniq=5034_318_6_85646_3&xpos=75.9&ypos=75.6&band=J&ra=211.3194905&dec=54.413845',
 'http://wsa.roe.ac.uk/cgi-bin/getFImage.cgi?file=/disk53/wsa/ingest/fits/20150125_v5/w20150125_02845_st.fit&mfid=8327217&extNo=4&lx=1275&hx=1425&ly=195&hy=345&rf=0&flip=1&uniq=5034_318_6_85646_4&xpos=76.1&ypos=75.6&band=J&ra=211.3194905&dec=54.413845',
 'http://wsa.roe.ac.uk/cgi-bin/getFImage.cgi?file=/disk53/wsa/ingest/fits/20150129_v5/w20150129_02901_st.fit&mfid=8278383&extNo=4&lx=1274&hx=1425&ly=195&hy=345&rf=0&flip=1&uniq=5034_318_6_85646_5&xpos=76.6&ypos=75.9&band=J&ra=211.3194905&dec=54.413845',
 'http://wsa.roe.ac.uk/cgi-bin/getFImage.cgi?file=/disk73/wsa/ingest/fits/20190614_v5/w20190614_00626_st.fit&mfid=11076607&extNo=4&lx=1276&hx=1426&ly=187&hy=337&rf=0&flip=1&uniq=5034_318_6_85646_6&xpos=75.9&ypos=75.7&band=K&ra=211.3194905&dec=54.413845']

This corresponds to this query on the web-form. On submiting the web-form query, it is clear that only these two files have deprecated=0. -

'http://wsa.roe.ac.uk/cgi-bin/getFImage.cgi?file=/disk53/wsa/ingest/fits/20150125_v5/w20150125_02715_st.fit&mfid=8327025&extNo=4&lx=1275&hx=1425&ly=195&hy=345&rf=0&flip=1&uniq=5034_318_6_85646_3&xpos=75.9&ypos=75.6&band=J&ra=211.3194905&dec=54.413845',
'http://wsa.roe.ac.uk/cgi-bin/getFImage.cgi?file=/disk73/wsa/ingest/fits/20190614_v5/w20190614_00626_st.fit&mfid=11076607&extNo=4&lx=1276&hx=1426&ly=187&hy=337&rf=0&flip=1&uniq=5034_318_6_85646_6&xpos=75.9&ypos=75.7&band=K&ra=211.3194905&dec=54.413845'

There doesn't seem to be any info about the deprecated status of an image from the URLs themselves.

@keflavich keflavich linked a pull request Aug 13, 2023 that will close this issue
Draft
@keflavich
Copy link
Contributor

Thanks @virajkaram , there's a straightforward enough way to get this metadata; see #2809 for the solution

@virajkaram
Copy link
Author

@keflavich awesome! Thank you!

@bsipocz bsipocz linked a pull request Aug 14, 2023 that will close this issue
Refactor WFAU image retrieval to reject deprecated images #2809
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
wfau
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Refactor WFAU image retrieval to reject deprecated images keflavich/astroquery
2 participants
@keflavich @virajkaram