OnlineStats.jl icon indicating copy to clipboard operation
OnlineStats.jl copied to clipboard

MovingTimeWindow

Open JoaoAparicio opened this issue 4 years ago • 3 comments

This[1] could be more efficient if the timestamps are assumed to be ordered. TimeSeries.jl enforces this. Do you think it would make sense to have some interop?

[1] https://github.com/joshday/OnlineStats.jl/blob/e554858134f38b3e22499b2de1e4be2e5eab3c67/src/stats/stats.jl#L394-L395

JoaoAparicio avatar Jul 05 '20 11:07 JoaoAparicio

That seems like a fair assumption. The _merge! method would need to change to handle the timestamp ordering.

What kind of interop are you thinking of?

joshday avatar Jul 05 '20 12:07 joshday

Currently we have function _fit!(o::MovingTimeWindow, y). In addition we could implement function _fit!(o::MovingTimeWindow, y::TimeArray). What is currently done is push all values of y into o, then filter. When the new _fit! method is called, timestamps in MovingTimeWindow are guaranteed to be sorted because timestamps in TimeArray are guaranteed to be sorted. So first compute the cutoff point, and then to determine what is kept and what is dropped, use searchsorted (if this point is in y, this is what from(y, cutoff) does, see [1]).

I'm sure this code can be made faster when we know the input array is time-sorted, I just don't know if the cleanest way to implement this is by adding function _fit!(o::MovingTimeWindow, y::TimeArray). Because then that would add TImeSeries.jl as a dependency to OnlineStats.jl, which is annoying.

[1] https://juliastats.org/TimeSeries.jl/latest/split/#from-1

JoaoAparicio avatar Jul 05 '20 13:07 JoaoAparicio

Sorry for the slow response! I'd be happy with changing MovingTimeWindow to assume all of the inputs are sorted. I agree with wanting to avoid a new dependency.

Is this something you'd like to implement? I'm about to have a baby (wife is due today!), so I'll be out of commission for a bit.

joshday avatar Jul 12 '20 19:07 joshday