JuliaTime
Support period range ::StepRange{NanoDate, <:Dates.Period} for NanoDate
julia> using NanoDates
julia> NanoDate("2022-10-27T00:00:00.000000000"):NanoDates.Second(10):NanoDate("2022-10-27T10:00:00.000000000")
ERROR: MethodError: no method matching zero(::Dates.CompoundPeriod)
Closest candidates are:
zero(::Union{Type{P}, P}) where P<:Dates.Period at ~/julia/usr/share/julia/stdlib/v1.8/Dates/src/periods.jl:53
zero(::AbstractIrrational) at irrationals.jl:150
zero(::CartesianIndex{N}) where N at multidimensional.jl:106
...
Stacktrace:
[1] steprange_last(start::NanoDate, step::Dates.Second, stop::NanoDate)
@ Dates ~/julia/usr/share/julia/stdlib/v1.8/Dates/src/ranges.jl:44
[2] StepRange
@ ./range.jl:316 [inlined]
[3] StepRange
@ ./range.jl:369 [inlined]
[4] _colon
@ ./range.jl:46 [inlined]
[5] (::Colon)(start::NanoDate, step::Dates.Second, stop::NanoDate)
@ Base ./range.jl:40
[6] top-level scope
@ none:1
Reasonable ask, somewhat involved implementation.
If you see no progress, please remind me in a month.
While reasonable, I have not addressed this request. The general case is rather involved. Is +seconds of the most import for you?
On Wed, May 3, 2023 at 2:44 PM Stas Gryumov @.***> wrote:
@JeffreySarnoff https://github.com/JeffreySarnoff Is there any progress?
— Reply to this email directly, view it on GitHub https://github.com/JuliaTime/NanoDates.jl/issues/29#issuecomment-1533529284, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM2VRV2LWRY2BFTTS5DCBDXEKRPNANCNFSM6AAAAAATZY4TGM . You are receiving this because you were mentioned.Message ID: @.***>
Ideally, I would like to use NanoDates instead of Dates in my work, so I would appreciate it if NanoDates could be as intuitive as Dates. Thank you! @JeffreySarnoff
If you allow that NanoDates is as intuitive as Dates with the exception of the omission of StepRange handling ... that is reason enough to fix it :).
Take this branch for a spin, let me know what works and if there are implementation gaps. NanoDates with ranges
I needed:
diff --git a/src/ranges.jl b/src/ranges.jl
index b87b0934..159c835c 100644
--- a/src/ranges.jl
+++ b/src/ranges.jl
@@ -67,5 +67,5 @@ end
Base.:(:)(a::NanoDate, b::NanoDate) = (:)(a, Day(1), b)
-guess(a::NanoDate, b::NanoDate, c) = floor(Int64, (Int128(value(b)) - Int128(value(a))) / tons(c))
-len(a::NanoDate, b::NanoDate, c) = Int64(div(value(b - a), tons(c)))
+Dates.guess(a::NanoDate, b::NanoDate, c) = floor(Int64, (Int128(value(b)) - Int128(value(a))) / tons(c))
+Dates.len(a::NanoDate, b::NanoDate, c) = Int64(div(value(b - a), tons(c)))
but with that this seems to work as expected—thanks very much!
There were a lot of warnings about broken precompilation because the convert methods in ranges.jl overwrite those in conversions.jl.
I did quickly profile range construction and calculation of a random element within the range, and it looks like lots of dynamic dispatch happens in various places. This leads to range construction taking more like 300 µs with NanoDates versus ~15 ns with DateTimes.
How did you "I did quickly profile range construction and calculation of a random element within the range, and it looks like lots of dynamic dispatch happens in various places."
Sorry, I should have said that.
I used this in VS Code:
julia> f(; n=1000) = begin
x = ndnow(UTC)
date = Date(0)
for i in 1:n
range = x:Millisecond(5):(x + Hour(1))
date = Date(range[end÷2])
end
date
end
f (generic function with 1 method)
julia> @profview f(; n=100_000) # Ignore this one; includes compilation
julia> @profview f(; n=100_000)
This yields the following flame graph, where red boxes indicate runtime dispatch:
and benchmarking:
julia> @benchmark f()
BenchmarkTools.Trial: 203 samples with 1 evaluation.
Range (min … max): 23.410 ms … 36.566 ms ┊ GC (min … max): 0.00% … 0.00%
Time (median): 23.810 ms ┊ GC (median): 0.00%
Time (mean ± σ): 24.717 ms ± 2.010 ms ┊ GC (mean ± σ): 2.49% ± 5.30%
▁▆█▅▃▁ ▂
██████▆█▁▇▁▁▄▁▆▄▄▁▁▁▄▁▁▁▄▁▁▁▁▁▁▁▁▇▇█▇▆▇▄▁▄▄▁▁▁▁▁▁▁▁▁▁▁▄▁▁▁▄ ▆
23.4 ms Histogram: log(frequency) by time 31.4 ms <
Memory estimate: 6.42 MiB, allocs estimate: 244013.
If you just do x = now() instead, you get this:
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
Range (min … max): 29.121 μs … 78.479 μs ┊ GC (min … max): 0.00% … 0.00%
Time (median): 29.263 μs ┊ GC (median): 0.00%
Time (mean ± σ): 29.328 μs ± 1.090 μs ┊ GC (mean ± σ): 0.00% ± 0.00%
▃▆▆▇▆█▃▁
▂▂▂▂▂▃▃▄▅▆█████████▆▅▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▁▁▂▂▂▁▂▂▂▂▂▂▁▂▂▁▁▂ ▃
29.1 μs Histogram: frequency by time 29.7 μs <
Memory estimate: 0 bytes, allocs estimate: 0.
Simple range construction timings:
julia> @btime x:Millisecond(5):(x + Hour(1)) setup=(x=DateTime(2000));
14.018 ns (0 allocations: 0 bytes)
julia> @btime x:Millisecond(5):(x + Hour(1)) setup=(x=NanoDate(2000));
15.870 μs (166 allocations: 4.50 KiB)
version 0.3.0 has merged your stepranges should work do me a favor and redo that profiling and looking for broken compilation
I see what is causing the time sink. It is fixable. Stand-by for v0.3.3.
v0.3.3 merged
using Dates, NanoDates
using BenchmarkTools;
BenchmarkTools.DEFAULT_PARAMETERS.evals = 1;
BenchmarkTools.DEFAULT_PARAMETERS.samples = 100_000;
BenchmarkTools.DEFAULT_PARAMETERS.time_tolerance = 1.0e-12;
dt1 = now() - Day(10) - Second(9999) - Millisecond(500);
dt2 = dt1 + Day(10) + Second(9999) + Millisecond(500);
dtspan = dt2 - dt1;
dtspan_periods = canonicalize(dtspan)
rem(dtspan.value, 4) == 0
dtstep = Millisecond(div(dtspan.value, 4))
dtsteprange = dt1:dtstep:dt2
dtcompoundstep = canonicalize(dtstep);
dtcompoundsteprange = dt1:dtcompoundstep:dt2
nd1 = NanoDate(dt1); nd2 = NanoDate(dt2); ndspan = nd2 - nd1;
canonical(ndspan)
canonicalize(ndspan)
ndstep = Nanosecond( NanoDates.tons(dtstep) )
ndsteprange = nd1:ndstep:nd2
ndcompoundstep = canonical(ndstep)
ndcompoundsteprange = nd1:ndcompoundstep:nd2
@btime collect($dtsteprange)
@btime collect($ndsteprange)
@btime collect($dtcompoundsteprange)
@btime collect($ndcompoundsteprange)
#=
julia> @btime collect($dtsteprange)
40.584 ns (1 allocation: 96 bytes)
5-element Vector{DateTime}:
2023-09-14T17:01:39.366
2023-09-17T05:43:19.241
2023-09-19T18:24:59.116
2023-09-22T07:06:38.991
2023-09-24T19:48:18.866
julia> @btime collect($ndsteprange)
48.785 ns (1 allocation: 144 bytes)
5-element Vector{NanoDate}:
2023-09-14T17:01:39.366
2023-09-17T05:43:19.241
2023-09-19T18:24:59.116
2023-09-22T07:06:38.991
2023-09-24T19:48:18.866
julia> @btime collect($dtcompoundsteprange)
98.000 μs (994 allocations: 25.11 KiB)
5-element Vector{DateTime}:
2023-09-14T17:01:39.366
2023-09-17T05:43:19.241
2023-09-19T18:24:59.116
2023-09-22T07:06:38.991
2023-09-24T19:48:18.866
julia> @btime collect($ndcompoundsteprange)
445.178 ns (7 allocations: 320 bytes)
5-element Vector{NanoDate}:
2023-09-14T17:01:39.366
2023-09-17T05:43:19.241
2023-09-19T18:24:59.116
2023-09-22T07:06:38.991
2023-09-24T19:48:18.866
julia> round(Int, NanoDates.tons(Microsecond(98)) / 445)
220
=#
for ranges with steps that are periods, Dates and NanoDates collect with similar rapidity.
for ranges with steps that are compoundperiods, at least for the above case, NanoDates collects two orders of magnitude faster than DateTime.
Amazing work, @JeffreySarnoff! This is great.
Per your request, here are the timings for f with ndnow above:
julia> @benchmark f()
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
Range (min … max): 61.070 μs … 131.018 μs ┊ GC (min … max): 0.00% … 0.00%
Time (median): 68.538 μs ┊ GC (median): 0.00%
Time (mean ± σ): 69.810 μs ± 12.161 μs ┊ GC (mean ± σ): 0.00% ± 0.00%
█ ▆ ▁ ▇▆▁▁▆▂▂ ▁ ▂▃▃▁ ▁▂▂▁ ▂
█▇██▅█▆███████▇█▇▅▃▇████▇▆▅▄▄▁▁▁▃▃▃▁▁▄▁▁▁▅▅▆▄▅▅▆▅▆▇████▇▆▇█▇ █
61.1 μs Histogram: log(frequency) by time 119 μs <
Memory estimate: 16 bytes, allocs estimate: 1.
The flamegraph from profiling looks like this: