pekko failed: MetricsBasedResizerSpec

refs: https://github.com/apache/pekko/actions/runs/13755814627/job/38463002991?pr=1702#step:7:5426

[03-10 03:03:26.191] [info] - must update the underutilizationStreak highestUtilization if current utilization is higher *** FAILED *** (1 millisecond)
[03-10 03:03:26.191] [info]   1 was not equal to 2 (MetricsBasedResizerSpec.scala:194)
[03-10 03:03:26.191] [info]   org.scalatest.exceptions.TestFailedException:
[03-10 03:03:26.191] [info]   at org.scalatest.matchers.MatchersHelper$.indicateFailure(MatchersHelper.scala:392)
[03-10 03:03:26.191] [info]   at org.scalatest.matchers.should.Matchers.shouldBe(Matchers.scala:7017)
[03-10 03:03:26.191] [info]   at org.scalatest.matchers.should.Matchers.shouldBe$(Matchers.scala:1808)
[03-10 03:03:26.191] [info]   at org.apache.pekko.testkit.PekkoSpec.shouldBe(PekkoSpec.scala:79)
[03-10 03:03:26.191] [info]   at org.apache.pekko.routing.MetricsBasedResizerSpec.f$proxy10$1(MetricsBasedResizerSpec.scala:194)
[03-10 03:03:26.191] [info]   at org.apache.pekko.routing.MetricsBasedResizerSpec.$init$$$anonfun$2$$anonfun$7(MetricsBasedResizerSpec.scala:184)
[03-10 03:03:26.191] [info]   at org.scalatest.Transformer.apply$$anonfun$1(Transformer.scala:22)
[03-10 03:03:26.191] [info]   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[03-10 03:03:26.191] [info]   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:31)
[03-10 03:03:26.191] [info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[03-10 03:03:26.191] [info]   at org.scalatest.Transformer.apply(Transformer.scala:22)
[03-10 03:03:26.191] [info]   at org.scalatest.Transformer.apply(Transformer.scala:21)
[03-10 03:03:26.191] [info]   at org.scalatest.wordspec.AnyWordSpecLike$$anon$3.apply(AnyWordSpecLike.scala:1118)
[03-10 03:03:26.191] [info]   at org.scalatest.TestSuite.withFixture(TestSuite.scala:196)
[03-10 03:03:26.191] [info]   at org.scalatest.TestSuite.withFixture$(TestSuite.scala:138)
[03-10 03:03:26.191] [info]   at org.apache.pekko.testkit.PekkoSpec.withFixture(PekkoSpec.scala:79)
[03-10 03:03:26.192] [info]   at org.scalatest.wordspec.AnyWordSpecLike.invokeWithFixture$1(AnyWordSpecLike.scala:1124)
[03-10 03:03:26.192] [info]   at org.scalatest.wordspec.AnyWordSpecLike.runTest$$anonfun$1(AnyWordSpecLike.scala:1128)
[03-10 03:03:26.192] [info]   at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
[03-10 03:03:26.192] [info]   at org.scalatest.wordspec.AnyWordSpecLike.runTest(AnyWordSpecLike.scala:1128)
[03-10 03:03:26.192] [info]   at org.scalatest.wordspec.AnyWordSpecLike.runTest$(AnyWordSpecLike.scala:44)
[03-10 03:03:26.192] [info]   at org.apache.pekko.testkit.PekkoSpec.runTest(PekkoSpec.scala:79)
[03-10 03:03:26.192] [info]   at org.scalatest.wordspec.AnyWordSpecLike.runTests$$anonfun$1(AnyWordSpecLike.scala:1187)
[03-10 03:03:26.192] [info]   at org.scalatest.SuperEngine.traverseSubNodes$1$$anonfun$1(Engine.scala:413)
[03-10 03:03:26.192] [info]   at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[03-10 03:03:26.192] [info]   at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[03-10 03:03:26.192] [info]   at scala.collection.immutable.List.foreach(List.scala:334)
[03-10 03:03:26.192] [info]   at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:429)
[03-10 03:03:26.192] [info]   at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:390)
[03-10 03:03:26.192] [info]   at org.scalatest.SuperEngine.traverseSubNodes$1$$anonfun$1(Engine.scala:427)
[03-10 03:03:26.192] [info]   at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[03-10 03:03:26.192] [info]   at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[03-10 03:03:26.192] [info]   at scala.collection.immutable.List.foreach(List.scala:334)
[03-10 03:03:26.192] [info]   at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:429)
[03-10 03:03:26.192] [info]   at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
[03-10 03:03:26.192] [info]   at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
[03-10 03:03:26.192] [info]   at org.scalatest.wordspec.AnyWordSpecLike.runTests(AnyWordSpecLike.scala:1187)
[03-10 03:03:26.192] [info]   at org.scalatest.wordspec.AnyWordSpecLike.runTests$(AnyWordSpecLike.scala:44)
[03-10 03:03:26.192] [info]   at org.apache.pekko.testkit.PekkoSpec.runTests(PekkoSpec.scala:79)
[03-10 03:03:26.192] [info]   at org.scalatest.Suite.run(Suite.scala:1114)
[03-10 03:03:26.192] [info]   at org.scalatest.Suite.run$(Suite.scala:564)
[03-10 03:03:26.192] [info]   at org.apache.pekko.testkit.PekkoSpec.org$scalatest$wordspec$AnyWordSpecLike$$super$run(PekkoSpec.scala:79)
[03-10 03:03:26.192] [info]   at org.scalatest.wordspec.AnyWordSpecLike.run$$anonfun$1(AnyWordSpecLike.scala:1232)
[03-10 03:03:26.192] [info]   at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
[03-10 03:03:26.193] [info]   at org.scalatest.wordspec.AnyWordSpecLike.run(AnyWordSpecLike.scala:1232)
[03-10 03:03:26.193] [info]   at org.scalatest.wordspec.AnyWordSpecLike.run$(AnyWordSpecLike.scala:44)
[03-10 03:03:26.193] [info]   at org.apache.pekko.testkit.PekkoSpec.org$scalatest$BeforeAndAfterAll$$super$run(PekkoSpec.scala:79)
[03-10 03:03:26.193] [info]   at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
[03-10 03:03:26.193] [info]   at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:217)
[03-10 03:03:26.193] [info]   at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:135)
[03-10 03:03:26.193] [info]   at org.apache.pekko.testkit.PekkoSpec.run(PekkoSpec.scala:79)
[03-10 03:03:26.193] [info]   at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
[03-10 03:03:26.193] [info]   at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
[03-10 03:03:26.193] [info]   at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
[03-10 03:03:26.193] [info]   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[03-10 03:03:26.193] [info]   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[03-10 03:03:26.193] [info]   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[03-10 03:03:26.193] [info]   at java.base/java.lang.Thread.run(Thread.java:829)

Mar 10 '25 06:03 He-Pin

This started failing a lot about 2 weeks ago.

In the Nightly tests, we run a matrix of plans with various Java and Scala versions so that same tests are run a lot. This test is almost bound to fail in at least one run now.

Mar 14 '25 09:03 pjfanning

Also seeing

must record the performance log with the correct pool size *** FAILED *** (1 millisecond)
[03-14 00:21:56.751] [info]   None was empty (MetricsBasedResizerSpec.scala:236)
[03-14 00:21:56.752] [info]   org.scalatest.exceptions.TestFailedException:
[03-14 00:21:56.752] [info]   at org.scalatest.matchers.MatchersHelper$.indicateFailure(MatchersHelper.scala:392)
[03-14 00:21:56.752] [info]   at org.scalatest.matchers.dsl.ResultOfNotWordForAny.be(ResultOfNotWordForAny.scala:670)
[03-14 00:21:56.753] [info]   at org.apache.pekko.routing.MetricsBasedResizerSpec.$anonfun$new$15(MetricsBasedResizerSpec.scala:236)

Failures are happening in 1.1 nightly tests so it seems like maybe this is just an issue with these tests as opposed to some new commit that has made them flaky.

Mar 14 '25 09:03 pjfanning

I found this gist that allows you to rerun each test multiple times. Even with bug numbers, the MetricsBasedResizerSpec tests are always pass for me locally. Maybe it is something about other tests, possibly run in parallel, that is affecting this test case.

Mar 23 '25 14:03 pjfanning

This suite has been flaky even in Akka days.

https://github.com/search?q=repo%3Aakka%2Fakka+MetricsBasedResizerSpec&type=issues

Not sure why it has become flakier recently. It fails for us with Pekko 1.0 and there are no recent changes there (much more regularly recently than a few months ago).

Mar 23 '25 22:03 pjfanning

https://github.com/apache/pekko/actions/runs/18805442582/job/53658867753?pr=2375#step:7:5009

Oct 25 '25 17:10 He-Pin

https://github.com/apache/pekko/actions/runs/18827956756/job/53713821643?pr=2402#step:7:4774

Oct 27 '25 07:10 He-Pin