hface icon indicating copy to clipboard operation
hface copied to clipboard

Integer overflow exception

Open royaldark opened this issue 6 years ago • 1 comments

Upon starting the hface server, it successfully connects to our Hazelcast cluster using the provided configuration, but the following exception is continually thrown:

java.lang.ArithmeticException: integer overflow
	at clojure.lang.Numbers.throwIntOverflow(Numbers.java:1501)
	at clojure.lang.Numbers.add(Numbers.java:1800)
	at clojure.lang.Numbers$LongOps.add(Numbers.java:455)
	at clojure.lang.Numbers.add(Numbers.java:128)
	at clojure.core$_PLUS_.invokeStatic(core.clj:973)
	at clojure.core$_PLUS_.invoke(core.clj:965)
	at clojure.core$merge_with$merge_entry__4881.invoke(core.clj:2954)
	at clojure.core$reduce1.invokeStatic(core.clj:925)
	at clojure.core$merge_with$merge2__4883.invoke(core.clj:2957)
	at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58)
	at clojure.core$reduce1.invokeStatic(core.clj:923)
	at clojure.core$reduce1.invokeStatic(core.clj:915)
	at clojure.core$merge_with.invokeStatic(core.clj:2950)
	at clojure.core$merge_with.doInvoke(core.clj:2942)
	at clojure.lang.RestFn.applyTo(RestFn.java:139)
	at clojure.core$apply.invokeStatic(core.clj:648)
	at clojure.core$apply.invoke(core.clj:641)
	at hface.stats$merge_stats$fn__1638.invoke(stats.clj:47)
	at hface.util$do_with_values$iter__362__366$fn__367.invoke(util.clj:47)
	at clojure.lang.LazySeq.sval(LazySeq.java:40)
	at clojure.lang.LazySeq.seq(LazySeq.java:49)
	at clojure.lang.RT.seq(RT.java:521)
	at clojure.core$seq__4357.invokeStatic(core.clj:137)
	at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:24)
	at clojure.core.protocols$fn__6738.invokeStatic(protocols.clj:75)
	at clojure.core.protocols$fn__6738.invoke(protocols.clj:75)
	at clojure.core.protocols$fn__6684$G__6679__6697.invoke(protocols.clj:13)
	at clojure.core$reduce.invokeStatic(core.clj:6545)
	at clojure.core$into.invokeStatic(core.clj:6610)
	at clojure.core$into.invoke(core.clj:6604)
	at hface.util$do_with_values.invokeStatic(util.clj:45)
	at hface.util$do_with_values.invoke(util.clj:44)
	at hface.stats$merge_stats.invokeStatic(stats.clj:47)
	at hface.stats$merge_stats.invoke(stats.clj:41)
	at hface.stats$aggregate_across_cluster$fn__1642.invoke(stats.clj:51)
	at clojure.core$map$fn__4785.invoke(core.clj:2644)
	at clojure.lang.LazySeq.sval(LazySeq.java:40)
	at clojure.lang.LazySeq.seq(LazySeq.java:49)
	at clojure.lang.RT.seq(RT.java:521)
	at clojure.core$seq__4357.invokeStatic(core.clj:137)
	at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:24)
	at clojure.core.protocols$fn__6738.invokeStatic(protocols.clj:75)
	at clojure.core.protocols$fn__6738.invoke(protocols.clj:75)
	at clojure.core.protocols$fn__6684$G__6679__6697.invoke(protocols.clj:13)
	at clojure.core$reduce.invokeStatic(core.clj:6545)
	at clojure.core$into.invokeStatic(core.clj:6610)
	at clojure.core$into.invoke(core.clj:6604)
	at hface.stats$aggregate_across_cluster.invokeStatic(stats.clj:50)
	at hface.stats$aggregate_across_cluster.invoke(stats.clj:49)
	at hface.stats$cluster_stats.invokeStatic(stats.clj:96)
	at hface.stats$cluster_stats.invoke(stats.clj:94)
	at hface.refresh$refresh_stats.invokeStatic(refresh.clj:50)
	at hface.refresh$refresh_stats.invoke(refresh.clj:47)
	at hface.refresh$collect_stats$fn__1714.invoke(refresh.clj:59)
	at hface.util$every$f__339.invoke(util.clj:13)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

It seems there are many keys in the grouped stats map (stats.clj:47) which are frequently very large (or very small) longs - such as :last-updated-time, :last-accessed-time, :created-time, :min-age, :max-age, etc.

I fixed this temporarily by replacing merge-with + with merge-with +' in stats.clj:47, but that seems like a pretty hacky solution - what does it even mean to sum the last updated times? Or sum the min ages?

Using latest hface and Hazelcast 3.8.6.

royaldark avatar Nov 06 '17 18:11 royaldark

interesting, I have not seen that, and you're right there is no real value in looking at aggregates for things like:

  • :last-updated-time
  • :last-accessed-time
  • :created-time
  • :min-age
  • :max-age
  • ..

a cleaner solution would of course be to ignore/filter out these unrelated fields from the aggregates.

But for now +' does the trick, and does not "harm" the aggregate view.

Let me know if you'd like to submit a PR, or I can do +' for now


while looking at this I upgraded hface and chazel to Hazelcast 3.9, in case you need the "latest stable".

tolitius avatar Nov 06 '17 19:11 tolitius