haxe.Timer not working when called from a thread without event loop

[acarioni]

Environment macos monterey 12.1 haxe 4.2.4

The class haxe.Timer throws the exception NoEventLoopException when it is called from a thread without an event loop. The error affects the threaded targets (I’ve tried with java, cs and python) and it is particularly insidious because haxe.Timer is part of the standard library and can be used not only by the user code but also by third-party libraries.

The issue can be reproduced by creating a program like this

package foo;

class Main {
  public static function main() {
    sys.thread.Thread.create(() -> haxe.Timer.delay(() -> trace("hello"), 0));
    Sys.sleep(1);
  }
}

The reported error (for the target java) is

Exception in thread "Thread-0" Event loop is not available. Refer to sys.thread.Thread.runWithEventLoop.
        at sys.thread.Thread$Thread_Impl_.get_events(/usr/local/lib/haxe/std/java/_std/sys/thread/Thread.hx:74)
        at haxe.Timer.new(/usr/local/lib/haxe/std/haxe/Timer.hx:76)
        at haxe.Timer.delay(/usr/local/lib/haxe/std/haxe/Timer.hx:141)
        at foo.Main$Closure_main_0.invoke(foo/Main.hx:5)
        at foo.Main$Closure_main_0.run(foo/Main.hx)
        at java.base/java.lang.Thread.run(Thread.java:834)
        at sys.thread.Thread$NativeHaxeThread.run(/usr/local/lib/haxe/std/java/_std/sys/thread/Thread.hx:166)

[haxiomic]

I think this is as expected – the critical question is which thread do you expect the timer callback to run on? The main thread, or your newly created thread? This of course has big implications for thread safety

I think it's expected to run on the newly created calling thread, but for that it requires an event loop

So the solution is to use Thread.runWithEventLoop rather than Thread.create, although perhaps we should make this more explicit for clarity in these situations, create() could be named createWithoutEventLoop() so it's not the obvious first choice

[acarioni]

My main objection is that in general a user doesn’t know (and should not know) on which thread the timer is going to be executed (and btw the documentation of haxe.Timer says nothing about event loops). Moreover threaded targets don’t even need an event loop to execute a timer because they can schedule the timer task in a background thread (as java.util.Timer does for example). As far as thread safety is concerned, this is of course a responsibility of the user and should be accomplished by means of locks or other synchronization mechanisms.

[acarioni]

As another example of why the actual implementation of Timer is a source of bugs in multithreaded applications, consider the following scenario that I’ve discovered a few days ago while I was making asynchronous tests by means of the library utest.

package foo;

class TestCase extends utest.Test {

  function testFoo(async: utest.Async) {
    veryComplexFunctionToBeTested(() -> {
      utest.Assert.pass();
      async.done();
    });
  }

  function testBar(async: utest.Async) {
    veryComplexFunctionToBeTested(() -> {
      utest.Assert.pass();
      async.done();
    });
  }

  function veryComplexFunctionToBeTested(callback: ()->Void) {
    sys.thread.Thread.create(callback);
  }
}

If you try to run the above code, you get

results: SOME TESTS FAILURES (success: false)
foo.TestCase
  testBar: ERROR E
    Event loop is not available. Refer to sys.thread.Thread.runWithEventLoop.
Called from sys.thread.Thread$Thread_Impl_.get_events (/Users/foobar/haxe/versions/4.2.4/std/java/_std/sys/thread/Thread.hx line 74)
Called from haxe.Timer.new (/Users/foobar/haxe/versions/4.2.4/std/haxe/Timer.hx line 76)
Called from haxe.Timer.delay (/Users/foobar/haxe/versions/4.2.4/std/haxe/Timer.hx line 141)
Called from foo.TestCase$Closure___initializeUtest___1.invoke (foo/TestCase.hx line 12)
Called from foo.TestCase$Closure___initializeUtest___1.invoke (foo/TestCase.hx line -1)

The problem, as you may guess, is that the library utest uses internally haxe.Timer to manage asynchronous tests and when the timer is called by a thread without event loop, the well-known error pops up.

[back2dos]

It is trivial for the user to create threads with an event loop if required. If they fail to do so, they will get very predictable errors. If instead we randomly dispatched timer tasks on some other thread, it may lead to race conditions that surface at the worst possible moment. That's a poor default.

If for some reason a users prefers Java's way of dispatching timer tasks, this can be accomplished by shadowing haxe.Timer from a userland class path. Probably not recommended though.

I would propose to update the documentation of Thread.create to state more clearly that haxe.Timer won't work.

[acarioni]

In the examples that I’ve shown, I spawned threads by means of Thread.create only to show as clearly as possible what was the problem.

But suppose that you pass a callback that creates a timer (maybe indirectly) to a third-party library: in that case you don’t have any control on the thread that eventually will execute the callback and consequently the timer. The bug that I found in the library utest highlights exactly this type of unpredictable behaviour: an asynchronous test fails notwithstanding it doesn’t use any timers.

For this reason in my opinion haxe.Timer should not depend on event loop machinery so that it works reliably in multi-threaded scenarios.

[back2dos]

But suppose that you pass a callback that creates a timer (maybe indirectly) to a third-party library: in that case you don’t have any control on the thread that eventually will execute the callback and consequently the timer.

I don't see that it matters. The call into said library is synchronous. The moment it creates a timer, you get a stack trace pointing back to your code (as seen in both your examples).

A different scenario would be a library that creates threads without run loops on which it runs user callbacks. That's questionable design at best. If I hand a callback onto a library, I expect it to call me back on the calling thread (unless it has extremely good reasons not to do so).

The bug that I found in the library utest highlights exactly this type of unpredictable behaviour: an asynchronous test fails notwithstanding it doesn’t use any timers.

To call this a bug is a bit of a stretch. Even so, the error is trivial to fix. In contrast, race conditions introduced by wildly dispatching user code on arbitrary threads created beyond the user's control are a hard thing to track down.

For this reason in my opinion haxe.Timer should not depend on event loop machinery so that it works reliably in multi-threaded scenarios.

Even if we assume you're "right", that's pretty irrelevant, because that's not something we can change just like that. It will silently break existing code that relies on the assumption that timers fire back on their original thread. I don't see that happening.

[RealyUniqueName]

I agree with Juraj's arguments. Also I think haxe.Timer should be as consistent across targets (even non-threaded ones) as possible. We should clarify documention of haxe.Timer and Thread.create about this case and maybe create sys.thread.Timer which would work similar to java's one.

[kLabz]

@Simn What more do you want to be done here?

[Simn]

Not sure, it mentioned event loops so I assigned you!

[kLabz]

Haha yeah I did check this issue while working on eval's event loop but it did seem to be working as intended

[kLabz]

Seems to me all that's left here is this:

maybe create sys.thread.Timer which would work similar to java's one.

Which is a feature we might not need [for 4.3]?

[Simn]

Fine for me, but let's at least update that notice on Timer, there's at least a typo there.

[kLabz]

What typo? :thinking:

[Simn]

Never mind, typo was in my head.