TimeZones.jl icon indicating copy to clipboard operation
TimeZones.jl copied to clipboard

Rework tests to only perform a single fetch data from IANA servers

Open omus opened this issue 2 years ago • 1 comments

The current TimeZones.jl setup fetches a specific version of the tzdata from the IANA services for running tests. When running on CI this data is always fetched at least once. It was noticed in #441 that we perform a second fetch from the IANA services in tzdata/build.jl which hasn't been problematic but causes unnecessary load on their servers.

We may also want to look into using GHA caching to avoid downloading from their servers entirely.

omus avatar Aug 22 '23 04:08 omus

Definitely should use some GHA caching:

[ Info: Loading tzdata 2016j
[ Info: Downloading tzdata 2016j archive
ERROR: LoadError: InitError: RequestError: Operation too slow. Less than 1 bytes/sec transferred the last 20 seconds while requesting https://data.iana.org/time-zones/releases/tzdata2016j.tar.gz

– https://github.com/JuliaTime/TimeZones.jl/actions/runs/5946262840/job/16126554201#step:6:131

Looks like the tzdata_version check may also be an isssue:

[ Info: Downloading Windows to POSIX timezone ID XML version: release-43-1
[ Info: Compiling Windows time zone name translation
[ Info: Loading tzdata 2016j
[ Info: Downloading tzdata 2016j archive
[ Info: Decompressing tzdata 2016j region data
[ Info: Compiling tzdata 2016j region data
tzdata_version: Error During Test at D:\a\TimeZones.jl\TimeZones.jl\test\tzdata\version.jl:57
  Got exception outside of a @test
  RequestError: HTTP/1.1 200 OK (Operation too slow. Less than 1 bytes/sec transferred the last 20 seconds) while requesting https://data.iana.org/time-zones/releases/
  Stacktrace:
    [1] (::Downloads.var"#9#18"{IOBuffer, Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Bool, String, Bool, Bool})(easy::Downloads.Curl.Easy)
      @ Downloads C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:388
    [2] with_handle(f::Downloads.var"#9#18"{IOBuffer, Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Bool, String, Bool, Bool}, handle::Downloads.Curl.Easy)
      @ Downloads.Curl C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Curl\Curl.jl:90
    [3] #8
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:329 [inlined]
    [4] arg_write(f::Downloads.var"#8#17"{Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Bool, String, Bool, Bool}, arg::IOBuffer)
      @ ArgTools C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\ArgTools\src\ArgTools.jl:134
    [5] #7
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:328 [inlined]
    [6] arg_read
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\ArgTools\src\ArgTools.jl:76 [inlined]
    [7] request(url::String; input::Nothing, output::IOBuffer, method::Nothing, headers::Vector{Pair{String, String}}, timeout::Float64, progress::Nothing, verbose::Bool, debug::Nothing, throw::Bool, downloader::Nothing)
      @ Downloads C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:327
    [8] request
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:295 [inlined]
    [9] #3
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:233 [inlined]
   [10] arg_write(f::Downloads.var"#3#4"{Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Nothing, String}, arg::IOBuffer)
      @ ArgTools C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\ArgTools\src\ArgTools.jl:134
   [11] #download#2
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:232 [inlined]
   [12] download
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:221 [inlined]
   [13] tzdata_versions()
      @ TimeZones.TZData D:\a\TimeZones.jl\TimeZones.jl\src\tzdata\download.jl:56
   [14] tzdata_latest_version()
      @ TimeZones.TZData D:\a\TimeZones.jl\TimeZones.jl\src\tzdata\download.jl:81
   [15] tzdata_version()
      @ TimeZones.TZData D:\a\TimeZones.jl\TimeZones.jl\src\tzdata\version.jl:92
   [16] (::var"#16#19")()
      @ Main D:\a\TimeZones.jl\TimeZones.jl\test\tzdata\version.jl:69
   [17] withenv(f::var"#16#19", keyvals::Pair{String, String})
      @ Base .\env.jl:197
   [18] macro expansion
      @ D:\a\TimeZones.jl\TimeZones.jl\test\tzdata\version.jl:68 [inlined]
   [19] macro expansion
...

– https://github.com/JuliaTime/TimeZones.jl/actions/runs/5946262840/job/16126554082#step:6:137

omus avatar Aug 23 '23 16:08 omus