staccato icon indicating copy to clipboard operation
staccato copied to clipboard

Creates batch adapter

Open tpitale opened this issue 9 years ago • 5 comments

  • Size-bounded queue using ruby stdlib SizedQueue for thread-safe queueing

TODO:

  • [x] test what net/http, faraday adapters do when given an array (must match batch body for GA)
  • [x] discuss clear behavior
  • [x] usage documentation
  • [x] update our usage of net/http to support array or string

Resolves #57

tpitale avatar Dec 02 '15 04:12 tpitale

@wjordan: Here's a quick spike. One issue (seemingly present in all the queueing/buffer code I looked at) is: how/when to trigger the "final" flushing of the buffer.

I've added a method called clear on the adapter, I could delegate to this from the tracker, but the application using this would have to somehow know when to call it. One example, a rails app, is probably not aware when the server crashes/quits. Events in this case would remain un-sent to GA. Seems less than ideal.

tpitale avatar Dec 02 '15 04:12 tpitale

@tpitale Awesome progress! This looks feasible. Flushing a partially-filled buffer can be handled a few different ways:

  1. Async flush operation the application can manually invoke to flush a partially-filled batch, e.g. at the end of a per-request ensure block (ruby-stud :force option)
  2. Configurable interval timer that automatically flushes a partially-filled buffer after a maximum delay
  3. Blocking flush operation the application can manually invoke to definitively flush the remaining buffered items e.g. on teardown, ensure block, etc. (ruby-stud :final option)

the last one is already covered by the existing clear method. For 1-2, we would need clear to support a partial flush without killing flushing_thread. This could be added to this PR by pushing a Null Object to the queue (to wake up the waiting flushing_thread), maybe something like:

FLUSH_OBJECT = Object.new.freeze
def clear(final=false)
  @queue << FLUSH_OBJECT
  if final
    @flushing_thread.kill
    @flushing_thread.join
    flush(true)
  end
end

def flush(final=false)
  param_array = []
  while param_array.size < @size && (object = @queue.deq(final)) != FLUSH_OBJECT
    param_array << object
  end
  rescue ThreadError
  ensure
    @adapter.post(param_array)
  end
end

wjordan avatar Dec 02 '15 17:12 wjordan

Okay, so you're thinking the use case would be to flush it at the end of a request, so that N events would get queued up and make a minimal number of calls to Google.

I think I can make some tweaks to make that work.

tpitale avatar Dec 03 '15 04:12 tpitale

@tpitale i commend how long you've been working on this thing! it has really help us out in our usecase (analytics for an iframe embed interactive widget, frontend analytics is being blocked)

sebbean avatar Sep 29 '20 05:09 sebbean

@wjordan @sebbean Are either of you using this? It has been sitting for a long time, and I'm not sure if anyone really needs it anymore …?

tpitale avatar Nov 27 '20 19:11 tpitale

Closing this as old and not sure it's necessary yet in V4.

tpitale avatar May 11 '23 15:05 tpitale