Cache timeout functionality

[jwoillez]

Currently configured by default to 24 hours (24*3600 seconds).

[keflavich]

OK, good idea and implementation so far. If we're going to have the cache timeout, though, we also need to have it purge - either automatically or after a notification to the user. Perhaps automatic is OK by default, but make it configurable.

[jwoillez]

I will add:

• an ESO override configuration item for this timeout,
• a disable capability specified with timeout = -1.
• and a flush capability at the module level.

@keflavich Could you safely keep a copy of your current astroquery.cfg? We need to port all the configuration items to the new default configuration file. I lost mine...

[jwoillez]

@keflavich Any idea where the travis failure comes from? It's limited to the egg_info, and Ihave no idea what they are for...

[keflavich]

Is it because you added the cfg to .gitignore?

[jwoillez]

Does not seem so. It is even more strange: ImportError: No module named astropy

[keflavich]

Huh... so that fix worked. Were we getting away with doing something very wrong before? Or perhaps it's just because you added astroquery.cfg

[keflavich]

Pristine (?) astroquery.cfg:

[besancon]

# Besancon download URL.  Changed to modele2003 in 2013.
# Options: ftp://sasftp.obs-besancon.fr/modele/modele2003/, ftp://sasftp.obs-besancon.fr/modele/
besancon_download_url = ftp://sasftp.obs-besancon.fr/modele/modele2003/

# Besancon model form URL
# Options: http://model.obs-besancon.fr/modele_form.php
besancon_model_Form = http://model.obs-besancon.fr/modele_form.php

# Amount of time before pinging the Besancon server to see if the file is
# ready.  Minimum 30s.
besancon_ping_delay = 30.0

# Timeout for Besancon query
besancon_timeout = 30.0

[eso]

# maximum number of rows returned (set to -1 for unlimited).
row_limit = 50

[fermi]

# Fermi query URL
# Options: http://fermi.gsfc.nasa.gov/cgi-bin/ssc/LAT/LATDataQuery.cgi
fermi_url = http://fermi.gsfc.nasa.gov/cgi-bin/ssc/LAT/LATDataQuery.cgi

# time limit for connecting to FERMI server
timeout = 60

# time limit for retrieving a data file once it has been located
retrieval_timeout = 120

[irsa]

# Name of the IRSA mirror to use.
# Options: http://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query
irsa_server = http://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query

# URL from which to list all the public catalogs in IRSA.
# Options: http://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-scan
gator_list_catalogs = http://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-scan

# maximum number of rows to retrieve in result
row_limit = 500

# time limit for connecting to the IRSA server
timeout = 60

[irsa_dust]

# Name of the irsa_dust server to use
# Options: http://irsa.ipac.caltech.edu/cgi-bin/DUST/nph-dust
irsa_dust_server = http://irsa.ipac.caltech.edu/cgi-bin/DUST/nph-dust

# default timeout for connecting to server
timeout = 30

[magpis]

# Name of the MAGPIS server.
# Options: http://third.ucllnl.org/cgi-bin/gpscutout
magpis_server = http://third.ucllnl.org/cgi-bin/gpscutout

# time limit for connecting to MAGPIS server
timeout = 60

[ned]

# Name of the NED mirror to use.
# Options: http://ned.ipac.caltech.edu/cgi-bin/
ned_server = http://ned.ipac.caltech.edu/cgi-bin/

# time limit for connecting to NED server
timeout = 60

# value of the Hubble Constant for many NED queries.
# Options: 73, 70.5
hubble_constant = 73

# the correct redshift for NED queries, see comments above.
# Options: 1, 2, 3, 4
correct_redshift = 1

# Frame in which to display the coordinates in the output.
# Options: Equatorial, Ecliptic, Galactic, SuperGalactic
output_coordinate_frame = Equatorial

# Equinox for the output coordinates.
# Options: J2000.0, B1950.0
output_equinox = J2000.0

# display output sorted by this criteria.
# Options: RA or Longitude, DEC or Latitude, GLON, GLAT, Redshift - ascending, Redshift - descending
sort_output_by = RA or Longitude

[nist]

# Name of the NIST URL to query.
# Options: http://physics.nist.gov/cgi-bin/ASD/lines1.pl
nist_server = http://physics.nist.gov/cgi-bin/ASD/lines1.pl

# time limit for connecting to NIST server
timeout = 30

[nrao]

# Name of the NRAO mirror to use.
# Options: https://archive.nrao.edu/archive/ArchiveQuery
nrao_server = https://archive.nrao.edu/archive/ArchiveQuery

# time limit for connecting to NRAO server
timeout = 60

[nvas]

# Name of the NVAS mirror to use.
# Options: https://webtest.aoc.nrao.edu/cgi-bin/lsjouwer/archive-pos.pl
nvas_server = https://webtest.aoc.nrao.edu/cgi-bin/lsjouwer/archive-pos.pl

# time limit for connecting to NVAS server
timeout = 60

[ogle]

# Name of the OGLE mirror to use.
# Options: http://ogle.astrouw.edu.pl/cgi-ogle/getext.py
ogle_server = http://ogle.astrouw.edu.pl/cgi-ogle/getext.py

# Time limit for connecting to the OGLE server.
timeout = 60

[sdss]

# Link to SDSS website.
sdss_server = http://data.sdss3.org/sas/dr10

# Max number of queries allowed per second
maxqueries = 1

# Default timeout for connecting to server
timeout = 30

[simbad]

# Name of the SIMBAD mirror to use.
# Options: simbad.u-strasbg.fr, simbad.harvard.edu
simbad_server = simbad.u-strasbg.fr

# time limit for connecting to Simbad server
timeout = 60

# maximum number of rows that will be fetched from the result.
row_limit = 0

[splatalogue]

Splatalogue SLAP interface URL (not used). = http://find.nrao.edu/splata-slap/slap

Splatalogue web interface URL. = http://www.cv.nrao.edu/php/splat/c_export.php

# default timeout for connecting to server
timeout = 60

Limit to number of lines exported = 10000

[template_module]

# put a brief description of the item here
# Options: http://dummy_server_mirror_1, http://dummy_server_mirror_2, http://dummy_server_mirror_n
server = http://dummy_server_mirror_1

# default timeout for connecting to server
timeout = 30

[ukidss]

# Name of the UKIDSS mirror to use.
# Options: http://surveys.roe.ac.uk:8080/wsa/
ukidss_server = http://surveys.roe.ac.uk:8080/wsa/

# time limit for connecting to UKIDSS server
timeout = 60

[vizier]

# Name of the VizieR mirror to use.
# Options: vizier.u-strasbg.fr, vizier.nao.ac.jp, vizier.hia.nrc.ca, vizier.ast.cam.ac.uk, vizier.cfa.harvard.edu, www.ukirt.jach.hawaii.edu, vizier.iucaa.ernet.in, vizier.china-vo.org
vizier_server = vizier.u-strasbg.fr

# default timeout for connecting to server
timeout = 60

# maximum number of rows that will be fetched from the result (set to -1 for
# unlimited).
row_limit = 50

[jwoillez]

Thanks for the "pristine" astroquery.cfg, will bundle this one with this PR.

Providing this file is a new requirement following the astropy configuration package update (see: http://docs.astropy.org/en/stable/config/config_0_4_transition.html#for-affiliated-package-authors).

Not a big issue, someone starting with the new configuration infrastructure would have been hard-pressed to find astroquery.cfg, not auto-generated anymore.

I still think that astropy should provide a way to auto-generate this template. Maybe a script ran by the packager...

[keflavich]

Let me know when this is ready. Since you haven't added the config, I assume it's not ready yet?

[keflavich]

@jwoillez I've merged in the latest astropy package-template, which seems to require a rebase on your part. I'll still wait for your go-ahead before a merge.

[jwoillez]

I completely forgot about this. Let me see what I can do... (hopefully soon).

[keflavich]

ping @jwoillez let me know if & when this is next ready for review

[kbruegge]

Any news on this? I think its a nice feature.

[bsipocz]

Closing this in favour of #1634 that has picked up changes from here and built on top of those.