cascalog icon indicating copy to clipboard operation
cascalog copied to clipboard

Overhead of push/pop thread bindings is high

Open ipostelnik opened this issue 9 years ago • 2 comments

My profiling showed that managing thread bindings for *op-call* and *flow-process* vars carries a lot of overhead. In my test these calls accounted for about 16% of flow execution, excluding IO (scheme input/task output). This was after I had applied the change from #283 to optimize null checks.

ipostelnik avatar May 18 '15 19:05 ipostelnik

@ipostelnik, yeah, this is a good catch. I wonder if we should set a flag in the config that would conditionally enable/disable this feature. Checking the config on each call would be far cheaper.

sritchie avatar May 19 '15 18:05 sritchie

#285 also relies on this functionality.

I guess we could add some sort of metadata annotation to operations that want to use these vars. Alternatively, maybe we could change them from proper clojure vars to something like a ThreadLocal?

ipostelnik avatar May 21 '15 15:05 ipostelnik