navo-workshop
navo-workshop copied to clipboard
Oddity with Vizier's tables endpoint in a new science use case
Mysterious issue with CDS tables metadata.
tablesV=registry.search(
servicetype='tap', keywords=['vizier'], includeaux=True
)[0].service.tables
print(f"Found {len(tablesV)} tables from Vizier")
for s in registry.search(
servicetype='tap', keywords=['heasarc'], includeaux=True
):
if 'heasarc' in s.ivoid:
tablesH=s.service.tables
break
print(f"Found {len(tablesH)} tables from HEASARC")
for t in tablesH:
print("works for HEASARC")
break
try:
for t in tablesV:
break
except Exception as e:
print(f"fails for Vizier:{e}")
gives
Found 47595 tables from Vizier
Found 1004 tables from HEASARC
works for HEASARC
fails for Vizier:1:0: no element found
Through the debugger, I see that the failure is coming from the fact that when you have an object that’s the result of asking for a tapservice.service.tables query, when you iterate over the tables, that goes into each pyvo.io.vosi.vodataservice.Table object and calls get_table_file() on the URL, e.g., http://tapvizier.u-strasbg.fr/TAPVizieR/tap/tables/J/AJ/144/129/refs and gets an error 200 in return. That in turn gets checked but passes through this try and you end up at line 168 with an empty response content and an exception.
163 try:
164 response.raise_for_status()
165 except requests.RequestException as ex:
166 raise DALServiceError.from_except(ex, tables_url)
167
168 -> table = vosi.parse_tables(response.raw.read).get_first_table()
169 self._cache[name] = table
And I get that 200 error for this table J/AJ/144/129 but not for another I randomly selected, http://tapvizier.u-strasbg.fr/TAPVizieR/tap/tables/j/aj/156/275/refs Seem to get the same thing with or without the ‘/refs’ at the end: a VOTtable that has the references for all the tables.
So a few questions:
- Why doesn't response.raise_for_status() not trap the 200 I get from the command-line?
- Why is something that is fetched for a specific table returning the same long thing for all tables in the case of Vizier?
- And why doesn't this work for Vizier?
I'll poke around a bit more when I have time, but figured I'd ask if anybody knows how this is supposed to work.