Runner offline with error: System.NullReferenceException: Object reference not set to an instance of an object.

Open kxxt opened this issue 3 months ago • 0 comments

First, thanks for your effort into making it work on riscv64.

I am using the latest release(v2.321.0) on Arch Linux RISC-V.

I noticed that while it works fine after setup, the runner goes offline after several hours with a NullReferenceException.

The log is here

[2025-10-11 12:31:53Z INFO MessageListener] Retriable exception: Object reference not set to an instance of an object.
[2025-10-11 12:31:53Z INFO RunnerServer] Refresh MessageQueue VssConnection to get on a different AFD node.
[2025-10-11 12:31:53Z INFO RunnerServer] EstablishVssConnection
[2025-10-11 12:31:53Z INFO RunnerServer] Establish connection with 60 seconds timeout.
[2025-10-11 12:31:53Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2025-10-11 12:31:55Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2025-10-11 12:31:55Z INFO MessageListener] Sleeping for 47.213 seconds before retrying.
[2025-10-11 12:32:42Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/kxxt/actions_runner/.credentials_rsaparams
[2025-10-11 12:32:42Z ERR  MessageListener] Catch exception during get next message.
[2025-10-11 12:32:42Z ERR  MessageListener] System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at GitHub.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at GitHub.Services.OAuth.VssOAuthTokenHttpClient.GetTokenAsync(VssOAuthGrant grant, VssOAuthClientCredential credential, VssOAuthTokenParameters tokenParameters, CancellationToken cancellationToken)
   at GitHub.Services.OAuth.VssOAuthTokenProvider.OnGetTokenAsync(IssuedToken failedToken, CancellationToken cancellationToken)
   at GitHub.Services.Common.IssuedTokenProvider.GetTokenOperation.GetTokenAsync(VssTraceActivity traceActivity)
   at GitHub.Services.Common.IssuedTokenProvider.GetTokenAsync(IssuedToken failedToken, CancellationToken cancellationToken)
   at GitHub.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at GitHub.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
   at GitHub.Runner.Listener.MessageListener.GetNextMessageAsync(CancellationToken token)
[2025-10-11 12:32:42Z INFO MessageListener] Retriable exception: Object reference not set to an instance of an object.
[2025-10-11 12:32:42Z INFO RunnerServer] Refresh MessageQueue VssConnection to get on a different AFD node.
[2025-10-11 12:32:42Z INFO RunnerServer] EstablishVssConnection
[2025-10-11 12:32:42Z INFO RunnerServer] Establish connection with 60 seconds timeout.
[2025-10-11 12:32:42Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2025-10-11 12:32:43Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2025-10-11 12:32:44Z INFO MessageListener] Sleeping for 59.778 seconds before retrying.
[2025-10-11 12:33:44Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/kxxt/actions_runner/.credentials_rsaparams
[2025-10-11 12:33:44Z ERR  MessageListener] Catch exception during get next message.
[2025-10-11 12:33:44Z ERR  MessageListener] System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at GitHub.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at GitHub.Services.OAuth.VssOAuthTokenHttpClient.GetTokenAsync(VssOAuthGrant grant, VssOAuthClientCredential credential, VssOAuthTokenParameters tokenParameters, CancellationToken cancellationToken)
   at GitHub.Services.OAuth.VssOAuthTokenProvider.OnGetTokenAsync(IssuedToken failedToken, CancellationToken cancellationToken)
   at GitHub.Services.Common.IssuedTokenProvider.GetTokenOperation.GetTokenAsync(VssTraceActivity traceActivity)
   at GitHub.Services.Common.IssuedTokenProvider.GetTokenAsync(IssuedToken failedToken, CancellationToken cancellationToken)
   at GitHub.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at GitHub.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
   at GitHub.Runner.Listener.MessageListener.GetNextMessageAsync(CancellationToken token)
[2025-10-11 12:33:44Z INFO MessageListener] Retriable exception: Object reference not set to an instance of an object.
[2025-10-11 12:33:44Z INFO RunnerServer] Refresh MessageQueue VssConnection to get on a different AFD node.
[2025-10-11 12:33:44Z INFO RunnerServer] EstablishVssConnection
[2025-10-11 12:33:44Z INFO RunnerServer] Establish connection with 60 seconds timeout.
[2025-10-11 12:33:44Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2025-10-11 12:33:45Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2025-10-11 12:33:45Z INFO MessageListener] Sleeping for 38 seconds before retrying.

Perhaps this is not a RISC-V related bug as there are similar upstream bug report like https://github.com/actions/runner/issues/3393 . Upstream has marked that bug as fixed but doesn't tell which version contains the fix. Maybe updating this fork to the latest upstream release could fix it.

Oct 11 '25 12:10 kxxt