Enable TAP for ESO module #3141

juanmcloaiza · 2024-11-28T16:54:56Z

Addresses #3138

Changes:

list_instruments
~~list_surveys~~ list_collections
query_instruments
~~query_surveys~~ query_collections
query_main
query_apex_quicklooks
deprecate functions *_surveys(...) in favor of _*collections(...)
Make sure that ALMA collection is handled properly

pep8speaks · 2024-11-28T16:55:02Z

Hello @juanmcloaiza! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2024-12-16 16:12:32 UTC

codecov · 2024-11-28T16:58:08Z

Codecov Report

Attention: Patch coverage is 46.34146% with 22 lines in your changes missing coverage. Please review.

Project coverage is 67.56%. Comparing base (50867f6) to head (da3b874).
Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
astroquery/eso/core.py	46.34%	22 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3141      +/-   ##
==========================================
+ Coverage   67.53%   67.56%   +0.02%     
==========================================
  Files         233      233              
  Lines       18487    18496       +9     
==========================================
+ Hits        12486    12497      +11     
+ Misses       6001     5999       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

bsipocz

One comment for now regarding the deprecation, otherwise is looking good so far (I didn't dive into the details, but had a quick look

bsipocz · 2024-11-28T17:02:36Z

astroquery/eso/core.py


 __doctest_skip__ = ['EsoClass.*']

+def decorator_with_params(dec):


There are deprecation decorators in astropy, is there a particular reason you choose not to use those?

Thanks for the info, I am now using them.

szampier · 2024-12-10T15:25:37Z

astroquery/eso/core.py

+            self._collection_list = list(res)
+        return self._collection_list
+
+    def _query_instrument_or_collection(self, query_on: QueryOnField, instmnt_or_clctn_name, *, column_filters={},


Mutable default arguments like [] or {} should be avoided. For such cases it's better to use None as default and initialise the mutable argument in the method, for example:
column_filters = column_filters or {}

True, thanks! 👍

szampier · 2024-12-10T15:27:36Z

astroquery/eso/core.py

+    USE_DEV_TAP = True
+
+    @staticmethod
+    def tap_url():


you could add a type hint for the return value

szampier · 2024-12-10T15:29:00Z

astroquery/eso/core.py


    def __init__(self):
        super().__init__()
        self._instrument_list = None
-        self._survey_list = None
+        self._collection_list = None


maybe it's a matter of taste but normally we don't need to specify the type in the variable name. In this case I would use collections instead of collection_list

Since all of those are initialized to None, maybe it would make sense to annotate the types instead of having "list" in the name?

The original name was _survey_list. My interpretation was that "list" refers not to a data type, but rather to a concept, like "The list of the supermarket" or "list of ingredients"; in this case a "List of available instruments/collections". Thus, I only changed the "survey" --> "collection", but I don't see disadvantages of removing _list from the name. Consider it done ;)

szampier · 2024-12-10T15:29:33Z

astroquery/eso/core.py

@@ -305,6 +175,24 @@ def _get_auth_header(self) -> Dict[str, str]:
        else:
            return {}

+    def _query_tap_service(self, query_str: str):


add type hint for return type

szampier · 2024-12-10T15:34:49Z

astroquery/eso/core.py

+        try:
+            table_to_return = tap.search(query_str).to_table()
+        except pyvo.dal.exceptions.DALQueryError:
+            raise pyvo.dal.exceptions.DALQueryError(f"\n\nError executing the following query:\n\n{query_str}\n\n")


Why rethrowing the same exception with a different error message? Is the original error message not clear enough? I think it's also possible to chain the original exception with a custom one but I don't remember the syntax

It is helpful to see the query that was being executed. In that way users can test it externally (for example here: http://archive.eso.org/programmatic/#TAP) and pinpoint the source of error. I found this to be the simplest way of achieving that goal, but I'm open to suggestions, if there's a better or standard way to do it.

One could wonder if this shouldn't be addressed in TAP itself -> after all TAP could return the query as part of the error message.

szampier · 2024-12-10T15:36:12Z

astroquery/eso/core.py

+            raise Exception(f"\n\nUnknown exception {e} while executing the following query: \n\n{query_str}\n\n")
+
+        return table_to_return
+
    def list_instruments(self, *, cache=True):


add type hints to method signature

szampier · 2024-12-10T16:06:56Z

astroquery/eso/core.py

+class QueryOnField():
+    table_name = ""
+    column_name = ""
+
+
+class QueryOnInstrument():
+    table_name = "dbo.raw"
+    column_name = "instrument"
+
+
+class QueryOnCollection():
+    table_name = "ivoa.ObsCore"
+    column_name = "obs_collection"
+


I think it would be cleaner to have one dataclass storing table and column information and three constants with the different copnfigurations, example:

@dataclass class QueryConfig: table_name: str column_name: str QUERY_ON_FIELD = QueryConfig(table_name="", column_name="") etc.

Absolutely, thanks! This is what I was looking for but didn't manage to remember how it was done properly. 👍

juanmcloaiza added 7 commits November 27, 2024 13:28

Switch to TAP list_surveys and list_instruments

3cd3b42

Define tap_url as static method

b74c1ca

Deselect ALMA at query level - TEMPORAL SOLUTION

d99d28c

Mark list_surveys as deprecated; define list_collections

2cbf6ea

Add name of the new function to deprecation warning

a022747

Deprecate list_ and query_ _surveys in favor of _collections

365b479

Replace 'survey' by 'collection' accross eso module

79ff276

bsipocz added the eso label Nov 28, 2024

bsipocz reviewed Nov 28, 2024

View reviewed changes

juanmcloaiza added 19 commits November 29, 2024 11:51

Make Pep8-compliant

7bea3b7

Use astroquery deprecated decorator

44b4553

Accept argument 'surveys=' when calling *_surveys() deprecated functions

9469279

Remove unused imports

26e2460

query_surveys --> query_collections in test_eso.py

613c870

Add CHANGES.rst

da3b874

Add test_tap_url()

10fc9d1

Remove hard-coded queries iside user-exposed functions

c00ee24

Remove function _activate_form

6b19cb5

Develop py2adql function with tests

6cd7c2a

Handle coord1 and coord2 arguments in query_collections

02706b3

Remove intersects arg from py2adql

6bbdfc3

Write TAP enabled query_collections; pass existing tests

b938d16

Remove intersects arg from py2adql test - not its responsibility

fd0859d

Add ATLASGAL to SGRA_COLLECTIONS

2db8538

Ignore DALOverflowWarning thrown on truncated long query results

9b6a4d8

Write test for func query_tap_service

d475cd4

Define and use _query_tap_service

08e8f56

fwd query_instrument and _collections to query_instrument_or_collection

88a334d

juanmcloaiza and others added 7 commits December 5, 2024 15:44

Remove unused code

11cc167

Take instrument or collection data from the corresponding table

06db98b

Ignore DALOverflowWarning on remote tests

56ef78d

Issue queries based on new QueryOnField class

fe00eb3

Remove unused function _query

6a37789

Modify test_multicollection to something less arbitrary

7dbd144

Make PEP8 linter happy

82141d8

szampier reviewed Dec 10, 2024

View reviewed changes

juanmcloaiza added 22 commits December 11, 2024 09:36

Replace mutable default arguments by None

be8ba36

Add type hints to query_instruments and query_collections

ac5aeb0

Rename *_list attributes; add return types

ac72d23

Add return types

f0496de

Refactor QueryOnField classes as dataclasses

a4ed293

correctly pass the dataclass to query_* function

ad92c7d

Annotate types of EsoClass.collections and instruments

ed78c6b

Add type hints to py2adql

fb7851f

Write utils function sanitize_val; Add top arg in py2adql

1021ab2

Write TAP version of query_main; sanitize constraints vals

42b3feb

correct typo in query string

2be3055

Add test_main_SgrAstar

6d63cd2

Row limit handled at query level. No filterwarnings needed

16d7bf7

query_main is always against dbo.raw

25afe1a

Add assert messages

da3c8c4

Make linter happy

07b1dd7

Add cache functionality to tap queries

5911797

Try mimicking the old warning behaviour

f70ca9c

Provisional fix to apex query

1a64bc9

Use t1.values_equal(t2) for table equality

dbaa676

Test table equality via t1.values_equal(t2)

82c0ae0

Do not exclude ALMA from collection queries

3ecc901

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable TAP for ESO module #3141

Enable TAP for ESO module #3141

juanmcloaiza commented Nov 28, 2024 •

edited

Loading

pep8speaks commented Nov 28, 2024 •

edited

Loading

codecov bot commented Nov 28, 2024 •

edited

Loading

bsipocz left a comment

bsipocz Nov 28, 2024

juanmcloaiza Nov 29, 2024

szampier Dec 10, 2024

juanmcloaiza Dec 11, 2024

szampier Dec 10, 2024

szampier Dec 10, 2024

Pharisaeus Dec 10, 2024

juanmcloaiza Dec 11, 2024

szampier Dec 10, 2024

szampier Dec 10, 2024

juanmcloaiza Dec 11, 2024 •

edited

Loading

Pharisaeus Dec 11, 2024

szampier Dec 10, 2024

szampier Dec 10, 2024

juanmcloaiza Dec 11, 2024


		__doctest_skip__ = ['EsoClass.*']

		def decorator_with_params(dec):

Enable TAP for ESO module #3141

Are you sure you want to change the base?

Enable TAP for ESO module #3141

Conversation

juanmcloaiza commented Nov 28, 2024 • edited Loading

Changes:

pep8speaks commented Nov 28, 2024 • edited Loading

Comment last updated at 2024-12-16 16:12:32 UTC

codecov bot commented Nov 28, 2024 • edited Loading

Codecov Report

bsipocz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

juanmcloaiza Dec 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

juanmcloaiza commented Nov 28, 2024 •

edited

Loading

pep8speaks commented Nov 28, 2024 •

edited

Loading

codecov bot commented Nov 28, 2024 •

edited

Loading

juanmcloaiza Dec 11, 2024 •

edited

Loading