navo-workshop icon indicating copy to clipboard operation
navo-workshop copied to clipboard

Oddity with Vizier's tables endpoint in a new science use case

Open trjaffe opened this issue 3 years ago • 0 comments

Mysterious issue with CDS tables metadata.

tablesV=registry.search(
    servicetype='tap', keywords=['vizier'], includeaux=True
    )[0].service.tables
print(f"Found {len(tablesV)} tables from Vizier")

for s in registry.search(
    servicetype='tap', keywords=['heasarc'], includeaux=True
    ):
    if 'heasarc' in s.ivoid:
        tablesH=s.service.tables
        break 
print(f"Found {len(tablesH)} tables from HEASARC")

for t in tablesH:
    print("works for HEASARC")
    break
try:
    for t in tablesV:
        break
except Exception as e:
    print(f"fails for Vizier:{e}")

gives

Found 47595 tables from Vizier
Found 1004 tables from HEASARC
works for HEASARC
fails for Vizier:1:0: no element found

Through the debugger, I see that the failure is coming from the fact that when you have an object that’s the result of asking for a tapservice.service.tables query, when you iterate over the tables, that goes into each pyvo.io.vosi.vodataservice.Table object and calls get_table_file() on the URL, e.g., http://tapvizier.u-strasbg.fr/TAPVizieR/tap/tables/J/AJ/144/129/refs and gets an error 200 in return. That in turn gets checked but passes through this try and you end up at line 168 with an empty response content and an exception.

 163                  try:
 164                      response.raise_for_status()
 165                  except requests.RequestException as ex:
 166                      raise DALServiceError.from_except(ex, tables_url)
 167
 168  ->                table = vosi.parse_tables(response.raw.read).get_first_table()
 169                  self._cache[name] = table

And I get that 200 error for this table J/AJ/144/129 but not for another I randomly selected, http://tapvizier.u-strasbg.fr/TAPVizieR/tap/tables/j/aj/156/275/refs Seem to get the same thing with or without the ‘/refs’ at the end: a VOTtable that has the references for all the tables.

So a few questions:

  1. Why doesn't response.raise_for_status() not trap the 200 I get from the command-line?
  2. Why is something that is fetched for a specific table returning the same long thing for all tables in the case of Vizier?
  3. And why doesn't this work for Vizier?

I'll poke around a bit more when I have time, but figured I'd ask if anybody knows how this is supposed to work.

trjaffe avatar Sep 01 '21 15:09 trjaffe