firebase-tools
firebase-tools copied to clipboard
!! functions: Failed to start functions in C:\functions: FirebaseError: Failed to load function.
[REQUIRED] Environment info
firebase-tools: 13.3.1
Platform: Windows
[REQUIRED] Test case
This is happening on any trigger event that tries to make more than 40+ updates
[REQUIRED] Steps to reproduce
"engines": { "node": "18" }, "dependencies": { "firebase": "^10.7.2", "firebase-admin": "^11.0.0", "firebase-functions": "^4.6.0" }
Start the emulator and the functions all load, the trigger functions work under small loads eg. 20 updates, but when I start pushing more updates through say 40-50 it throws the following error.
[REQUIRED] Expected behavior
Expect the trigger function to run without throwing errors.
[REQUIRED] Actual behavior
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by io.netty.util.internal.ReflectionUtil (file:/C:/Users/Adam/.cache/firebase/emulators/firebase-database-emulator-v4.11.2.jar) to field sun.nio.ch.SelectorImpl.selectedKeys
WARNING: Please consider reporting this to the maintainers of io.netty.util.internal.ReflectionUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
13:05:48.282 [NamespaceSystem-akka.actor.default-dispatcher-6] INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
13:05:48.541 [main] INFO com.firebase.server.forge.App$ - Listening at 127.0.0.1:9000
13:05:49.394 [NamespaceSystem-akka.actor.default-dispatcher-7] INFO com.firebase.core.namespace.NamespaceActor - fantasykeeperleague-nfikl-default-rtdb successfully activated FBKV (SurveyIdle(0)) wait: 287ms, init: 0ms
13:06:15.965 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:15.984 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:15.996 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.008 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.018 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.030 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.041 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.051 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.061 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.073 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.083 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.093 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.105 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.114 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.148 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.161 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.175 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.190 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.210 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.231 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:06:16.394 [NamespaceSystem-blocking-namespace-operation-dispatcher-12] INFO com.firebase.core.namespace.StateManager - Namespace fantasykeeperleague-nfikl-default-rtdb status Active to Active
13:07:42.854 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"XZ0N1va9+DqedQtpiEbiKEohUwQ=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/19/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.854 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"UbBglkCIa6YpiawUjUMPSM9loE8=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/23/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.854 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"cxLso2C4CtSEFt8b4mhh6A/Xrwk=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/44/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.854 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"LGa+FDhY3kcC3CB5WpEqV04cvWI=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/33/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.856 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"Dj5VMySf8aOPSwB22DC7T+SNWwA=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/50/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.858 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"BxOfzBW5o7dv//7Z4VURzkvdfeg=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/37/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.859 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"FuCqKeKfX/ZpX8s/KOQHZ1uroPU=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/13/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.860 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"4Bk9IpzcKdJjydjZy807MtsYzhg=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/46/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.861 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"7HqrYmNRfHEd+XlNAxh/f7NKpiY=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/35/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.862 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"WzaHsHnUh3jwZuIOuCcj1vKEZI0=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/16/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.862 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"P1cvXzKgnD/4wGhjseheiN/Y7DI=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/5/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.862 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"gFF90deBdwa1v+Ud9IalGReouwc=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/10/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.863 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"liblpnM4BXYnIBiGHrk847Lp7FA=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/21/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.863 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"/DZVQ/NuNkmZ/STjpZT5juagc5g=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/32/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.863 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"6/d7PcC8AM9OTGrDPNslIMkjsQs=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/49/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.866 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"zx+mkA7UXbMtoHKvs5Siahiz7Y0=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/39/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.867 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"neNSYoijptEHKGHO7Xdnn7epBMk=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/2/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.868 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"yRzexEo+/d9Ljj1dn7w4kn+z8LE=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/18/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.869 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"WQZZX/btH5KnN6GYVnnQ5ruz6mM=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/30/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:42.870 [NamespaceSystem-blocking-namespace-operation-dispatcher-18] INFO com.firebase.core.stats.EventAuditLog - {"eventId":"7LWkMw898CsH6jzGb1QUAWTTwy0=","namespace":"fantasykeeperleague-nfikl-default-rtdb","projectNumber":"123456789","eventType":"providers/google.firebase.database/eventTypes/ref.update","flowName":"projects/fantasykeeperleague-nfikl/topics/us-central1-aflGamesIntoPlayerViewOnUpdate-0","resource":"projects//instances/fantasykeeperleague-nfikl-default-rtdb/refs/PLAYERS/3/PROFILE/aflGames","evaluationTimestamp":"2024-02-23T02:07:42.751Z","stage":"TRIGGERED"}
13:07:52.947 [TriggersWorkerPool-3-1] ERROR com.firebase.server.client.GCFEmulatorPublisher - fantasykeeperleague-nfikl-default-rtdb failed to publish trigger to functions emulator with exception
com.firebase.server.router.IdleTimeoutException: null
at com.firebase.server.router.IdleStateDetectionHandler$.
Removing the functions tag since this is an exception being thrown in the database emulator.
are you sure it's not related to emulator functions?
1:33:50.275 [TriggersWorkerPool-3-1] ERROR com.firebase.server.client.GCFEmulatorPublisher - fantasykeeperleague-75d13 failed to publish trigger to functions emulator with exception\r\ncom.firebase.server.router.IdleTimeoutException: null\r\n\tat com.firebase.server.router.IdleStateDetectionHandler$.
Since the stack trace is in a different language than the functions emulator, I'm pretty sure :wink:
Your use case is certainly related to being a functions user, but the realtime database team owns the realtime database emulator, including its code to publish events that the functions emulator then serves. I didn't remove the tag to say it was unimportant, only to control which team's inbox reports the bug. I can see who is currently responsible for looking at this.
The stacktrace is in RTDB emulator. GCFEmulatorPublisher
is the client connecting to functions emulator and try to publish any events. It's throwing the exception because the connection to function emulator timed out.
There could be a bug in either GCF emulator or RTDB emulator's connection protocol.
Can we reproduce this bug and also include logs and analysis on the functions emulator?
We should have more insights on the functions emulator isn't responding to RTDB emulator's publish request.
Is Akka single-threading all responses to the emulator? Since we don't have a Pub/Sub emulator to intermediate RTDB -> GCF emulators, the RTDB may need to handle potentially unbounded queuing, especially in cases where the customer runs --inspect-functions
where
- The requests get queued to a single process
- The process may hang indefinitely when a debugger is attached.