Skip to content

Commit

Permalink
Updated docs for mast_query feature; new changelong; fix syntax
Browse files Browse the repository at this point in the history
  • Loading branch information
@mstcyr2 @bsipocz
mstcyr2 authored and bsipocz committed Jan 28, 2024
1 parent 4d15ee9 commit fa099a6
Show file tree
Hide file tree
Showing 4 changed files with 189 additions and 33 deletions.
3 changes: 3 additions & 0 deletions CHANGES.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,9 @@ mast

- Bug fix in ``Observations.query_criteria()`` to use ``page`` and ``pagesize`` parameters [#2915]

- Added ``Mast.mast_query`` to ``MastClass`` to handle the creation of parameter dictionaries for
MAST Service queries. [#2785]

nist
^^^^

Expand Down
40 changes: 33 additions & 7 deletions astroquery/mast/observations.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -909,35 +909,61 @@ def service_request_async(self, service, params, *, pagesize=None, page=None, **

return self._portal_api_connection.service_request_async(service, params, pagesize, page, **kwargs)

def mast_query(self, service, columns='*', **kwargs):
def mast_query(self, service, columns=None, **kwargs):
"""
Given a Mashup service and parameters as keyword arguments, builds and excecutes a Mashup query.
Parameters
----------
service : str
The Mashup service to query.
columns : str, optional
Specifies the columns to be returned as a comma-separated list, e.g. "ID, ra, dec".
**kwargs :
Service-specific parameters and MashupRequest properties. See the
`service documentation <https://mast.stsci.edu/api/v0/_services.html>`__ and the
`MashupRequest Class Reference <https://mast.stsci.edu/api/v0/class_mashup_1_1_mashup_request.html>`__
for valid keyword arguments.
Returns
-------
response : `~astropy.table.Table`
"""
# Specific keywords related to positional and MashupRequest parameters.
position_keys = ['ra', 'dec', 'radius', 'position']
request_keys = ['format', 'data', 'filename', 'timeout', 'clearcache',
'removecache', 'removenullcolumns', 'page', 'pagesize']

# Explicit formatting for Mast's filtered services
if 'Filtered' in service:
if 'filtered' in service.lower():

# Separating the filter params from the positional and service_request method params.
filters = [{'paramName': k, 'values': kwargs[k]} for k in kwargs
if k not in position_keys+request_keys]
position_params = {k: v for k, v in kwargs.items() if k in position_keys}
request_params = {k: v for k, v in kwargs.items() if k in request_keys}
if k.lower() not in position_keys+request_keys]
position_params = {k: v for k, v in kwargs.items() if k.lower() in position_keys}
request_params = {k: v for k, v in kwargs.items() if k.lower() in request_keys}

# Mast's filtered services require at least one filter
if filters == []:
raise InvalidQueryError("Please provide at least one filter.")

# Building 'params' for Mast.service_request
if columns is None:
columns = '*'

params = {'columns': columns,
'filters': filters,
**position_params
}
else:

# Separating service specific params from service_request method params
params = {k: v for k, v in kwargs.items() if k not in request_keys}
request_params = {k: v for k, v in kwargs.items() if k in request_keys}
params = {k: v for k, v in kwargs.items() if k.lower() not in request_keys}
request_params = {k: v for k, v in kwargs.items() if k.lower() in request_keys}

# Warning for wrong input
if columns is not None:
warnings.warn("'columns' parameter will not mask non-filtered services", InputWarning)

return self.service_request(service, params, **request_params)

Expand Down
10 changes: 9 additions & 1 deletion astroquery/mast/tests/test_mast.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -309,9 +309,17 @@ def test_mast_query(patch_post):
dataproduct_type=['image'],
proposal_pi=['Osten, Rachel A.'],
s_dec=[{'min': 43.5, 'max': 45.5}])
pp_list = result['proposal_pi']
sd_list = result['s_dec']
assert isinstance(result, Table)
assert len(set(pp_list)) == 1
assert max(sd_list) < 45.5
assert min(sd_list) > 43.5

# filtered search with position
# error handling
with pytest.raises(InvalidQueryError) as invalid_query:
mast.Mast.mast_query('Mast.Caom.Filtered')
assert "Please provide at least one filter." in str(invalid_query.value)


def test_resolve_object(patch_post):
Expand Down
169 changes: 144 additions & 25 deletions docs/mast/mast_mastquery.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,127 @@
MAST Queries
************

Direct Mast Queries
===================

The Mast class provides more direct access to the MAST interface. It requires
more knowledge of the inner workings of the MAST API, and should be rarely
needed. However in the case of new functionality not yet implemented in
astroquery, this class does allow access. See the `MAST api documentation
<https://mast.stsci.edu/api>`_ for more information.
astroquery, this class does allow access. See the
`MAST api documentation <https://mast.stsci.edu/api>`__ for more
information.

The basic MAST query function allows users to query through the following
`MAST Services <https://mast.stsci.edu/api/v0/_services.html>`__ using
their corresponding parameters and returns query results as an
`~astropy.table.Table`.

Filtered Mast Queries
=====================

MAST's Filtered services use the parameters 'columns' and 'filters'. The 'columns'
parameter is a required string that specifies the columns to be returned as a
comma-separated list. The 'filters' parameter is a required list of filters to be
applied. The `~astroquery.mast.MastClass.mast_query` method accepts that list of
filters as keyword arguments paired with a list of values, similar to
`~astroquery.mast.ObservationsClass.query_criteria`.

The following example uses a JWST service with column names and filters specific to
JWST services. For the full list of valid parameters view the
`JWST Field Documentation <https://mast.stsci.edu/api/v0/_jwst_inst_keywd.html>`__.

.. doctest-remote-data::

>>> from astroquery.mast import Mast
...
>>> observations = Mast.mast_query('Mast.Jwst.Filtered.Nirspec',
... columns='title, instrume, targname',
... targoopp=['T'])
>>> print(observations) # doctest: +IGNORE_OUTPUT
title instrume targname
------------------------------- -------- ----------------
ToO Comet NIRSPEC ZTF (C/2022 E3)
ToO Comet NIRSPEC ZTF (C/2022 E3)
ToO Comet NIRSPEC ZTF (C/2022 E3)
ToO Comet NIRSPEC ZTF (C/2022 E3)
De-Mystifying SPRITEs with JWST NIRSPEC SPIRITS18nu
ToO Comet NIRSPEC ZTF (C/2022 E3)
... ... ...
ToO Comet NIRSPEC ZTF (C/2022 E3)
ToO Comet NIRSPEC ZTF (C/2022 E3)
ToO Comet NIRSPEC ZTF (C/2022 E3)
Length = 319 rows


TESS Queries
------------

The basic MAST query function returns query results as an `~astropy.table.Table`.
TESS queries have 2 types of filtered services. To output a table and specify
columns for a TESS query, use TIC or CTL services with '.Rows' on the end
(e.g. `Mast.Catalogs.Filtered.Tic.Rows
<https://mast.stsci.edu/api/v0/_services.html#MastCatalogsFilteredTicRows>`__).
Valid parameters for TIC and CTL services are detailed in the
`TIC Field Documentation <https://mast.stsci.edu/api/v0/_t_i_cfields.html>`__.

.. doctest-remote-data::

>>> from astroquery.mast import Mast
...
>>> service = 'Mast.Caom.Cone'
>>> params = {'ra':184.3,
... 'dec':54.5,
... 'radius':0.2}
>>> observations = Mast.service_request(service, params)
>>> observations = Mast.mast_query('Mast.Catalogs.Filtered.Tic.Rows',
... columns='id',
... dec=[{'min': -90, 'max': -30}],
... Teff=[{'min': 4250, 'max': 4500}],
... logg=[{'min': 4.5, 'max': 5.0}],
... Tmag=[{'min': 8, 'max': 10}])
>>> print(observations) # doctest: +IGNORE_OUTPUT
ID
---------
320274328
408290683
186485314
395586623
82007673
299550797
...
333372236
394008846
261525246
240766734
240849919
219338557
92131304
Length = 814 rows

TESS services without '.Rows' in the title are used for count queries and will
not mask the output tables using the columns parameter. Additionally, using a
'.Rows' service for a count query will result in an error.

.. doctest-skip::

>>> from astroquery.mast import Mast
...
>>> observations = Mast.mast_query('Mast.Catalogs.Filtered.Tic.Rows',
... columns = 'COUNT_BIG(*)',
... dec=[{'min': -90, 'max': -30}],
... Teff=[{'min': 4250, 'max': 4500}],
... logg=[{'min': 4.5, 'max': 5.0}],
... Tmag=[{'min': 8, 'max': 10}])
Traceback (most recent call last):
...
astroquery.exceptions.RemoteServiceError: Incorrect syntax near '*'.


Cone Searches
=============

MAST's cone search services use the parameters 'ra', 'dec', and 'radius' and return
a table of observations with all columns present.

.. doctest-remote-data::

>>> from astroquery.mast import Mast
...
>>> observations = Mast.mast_query('Mast.Caom.Cone',
... ra=184.3,
... dec=54.5,
... radius=0.2)
>>> print(observations) # doctest: +IGNORE_OUTPUT
intentType obs_collection provenance_name ... obsid distance
---------- -------------- --------------- ... ----------- ------------------
Expand Down Expand Up @@ -51,28 +152,46 @@ The basic MAST query function returns query results as an `~astropy.table.Table`
Length = 77 rows


Many mast services, specifically JWST and Catalog services, require the two principal keywords, 'columns' and 'filters',
to list parameters. Positional services will also require right ascension and declination parameters, either in
addition to columns and filters or on their own. For example, the cone search service only requires the 'ra' and
'dec' parameters. Using the wrong service parameters will result in an error. Read the
`MAST API services documentation <https://mast.stsci.edu/api/v0/_services.html>`__ for more information on valid
service parameters.
Cone search services only require positional parameters. Using the wrong service
parameters will result in an error. Read the
`MAST API services documentation <https://mast.stsci.edu/api/v0/_services.html>`__
for more information on valid service parameters.

.. doctest-remote-data::
.. doctest-skip::

>>> from astroquery.mast import Mast
...
>>> service = 'Mast.Caom.Cone'
>>> params = {'columns': "*",
... 'filters': {}}
>>> observations = Mast.service_request(service, params)
>>> observations = Mast.mast_query('Mast.Caom.Cone',
... columns='ra',
... Teff=[{'min': 4250, 'max': 4500}],
... logg=[{'min': 4.5, 'max': 5.0}])
Traceback (most recent call last):
...
astroquery.exceptions.RemoteServiceError: Request Object is Missing Required Parameter : RA

Using the 'columns' parameter in addition to the required cone search parameters will
result in a warning.

.. doctest-remote-data::

>>> from astroquery.mast import Mast
...
>>> observations = Mast.mast_query('Mast.Catalogs.GaiaDR1.Cone',
... columns="ra",
... ra=254.287,
... dec=-4.09933,
... radius=0.02) # doctest: +SHOW_WARNINGS
InputWarning: 'columns' parameter will not mask non-filtered services

Advanced Service Request
========================

If the output is not the MAST json result type it cannot be properly parsed into a `~astropy.table.Table`.
In this case, the async method should be used to get the raw http response, which can then be manually parsed.
Certain MAST Services, such as `Mast.Name.Lookup
<https://mast.stsci.edu/api/v0/_services.html#MastNameLookup>`__ will not work with
`astroquery.mast.MastClass.mast_query` due to it's return type. If the output of a query
is not the MAST json result type it cannot be properly parsed into a `~astropy.table.Table`.
In this case, the `~astroquery.mast.MastClass.service_request_async` method should be used
to get the raw http response, which can then be manually parsed.

.. doctest-remote-data::

Expand All @@ -82,7 +201,7 @@ In this case, the async method should be used to get the raw http response, whic
>>> params ={'input':"M8",
... 'format':'json'}
...
>>> response = Mast.service_request_async(service,params)
>>> response = Mast.service_request_async(service, params)
>>> result = response[0].json()
>>> print(result) # doctest: +IGNORE_OUTPUT
{'resolvedCoordinate': [{'cacheDate': 'Apr 12, 2017 9:28:24 PM',
Expand Down

0 comments on commit fa099a6

Please sign in to comment.