Broken automatic update not taking node version update into account (node20 -> node24), with cascading effects... #4064
Description
With self-hosted runner agent deployed in version 2.321.0 (node20), the automatic update to 2.328.0 doesn't self-updates to support and use node24. This results in github actions relying on node24 to fail later when such job is run. It may be that the automatic update process is not working (see logs below).
To Reproduce
Steps to reproduce the behavior:
- install a self-hosted runner with a v2.321.0 agent (keep automatic update enabled)
- start the agent
- let it auto update
- start a workflow with a job using latest version of actions/checkout:
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0Expected behavior
The job does not fail as the automatic update to v2.328.0 makes the support of node24 work.
Runner Version and Platform
v2.321.0 automatically updated to v2.328.0 (at least that's what it looks like)
Ubuntu 24.04
What's not working?
Job seems to be running still in v2.321.0, also node24 is not supported resulting in other github actions failing.
Job Log Output
Runner and Worker's Diagnostic Logs
SelfUpdate-20251001-052936.log.succeed:
[2025-10-01 05:29:36-6927] --------whoami--------
ubuntu
[2025-10-01 05:29:36-6947] --------whoami--------
[2025-10-01 05:29:36-6957] Waiting for Runner.Listener (2346) to complete
[2025-10-01 05:29:36-6967] Process 2346 still running
[2025-10-01 05:29:38-0015] Process 2346 finished running
[2025-10-01 05:29:38-0025] Sleep 1 more second to make sure process exited
[2025-10-01 05:29:39-0015] move /home/ubuntu/actions-runner/bin /home/ubuntu/actions-runner/bin.2.321.0
renamed '/home/ubuntu/actions-runner/bin' -> '/home/ubuntu/actions-runner/bin.2.321.0'
[2025-10-01 05:29:39-0040] move /home/ubuntu/actions-runner/externals /home/ubuntu/actions-runner/externals.2.321.0
renamed '/home/ubuntu/actions-runner/externals' -> '/home/ubuntu/actions-runner/externals.2.321.0'
[2025-10-01 05:29:39-0062] Create junction bin folder
[2025-10-01 05:29:39-0082] Create junction externals folder
[2025-10-01 05:29:39-0127] Update runsvc.sh
[2025-10-01 05:29:39-0147] Update succeed
[2025-10-01 05:29:39-0165] update.finished file creation succeed
[2025-10-01 05:29:39-0175] Rename /home/ubuntu/actions-runner/_diag/SelfUpdate-20251001-052936.log to be /home/ubuntu/actions-runner/_diag/SelfUpdate-20251001-052936.log.succeed
renamed '/home/ubuntu/actions-runner/_diag/SelfUpdate-20251001-052936.log' -> '/home/ubuntu/actions-runner/_diag/SelfUpdate-20251001-052936.log.succeed'
Runner_20251001-050542-utc.log:
[2025-10-01 05:05:46Z INFO Runner] Refresh message received, kick-off selfupdate background process.
[2025-10-01 05:05:46Z INFO SelfUpdater] Version '2.328.0' of 'agent' package available in server.
[2025-10-01 05:05:46Z INFO SelfUpdater] Current running runner version is 2.321.0
[2025-10-01 05:05:46Z INFO SelfUpdater] An update is available.
[2025-10-01 05:05:46Z INFO Terminal] WRITE LINE: Runner update in progress, do not shutdown runner.
[2025-10-01 05:05:46Z INFO SelfUpdater] RunnerPlatform: linux-x64
[2025-10-01 05:05:46Z INFO Terminal] WRITE LINE: Downloading 2.328.0 runner
[2025-10-01 05:05:47Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:05:47Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:05:47Z INFO HostContext] Well known directory 'Work': '/home/ubuntu/actions-runner/_work'
[2025-10-01 05:05:47Z INFO SelfUpdater] Attempt 1: save latest runner into /home/ubuntu/actions-runner/_work/_update/runner1.tar.gz.
[2025-10-01 05:05:47Z INFO SelfUpdater] Download runner: begin download
[2025-10-01 05:05:47Z INFO SelfUpdater] Downloading https://github.com/actions/runner/releases/download/v2.328.0/actions-runner-linux-x64-2.328.0.tar.gz
[2025-10-01 05:05:54Z INFO SelfUpdater] Download runner: finished download
[2025-10-01 05:05:54Z INFO SelfUpdater] Validated Runner Hash matches /home/ubuntu/actions-runner/_work/_update/runner1.tar.gz : 01066fad3a2893e63e6ca880ae3a1fad5bf9329d60e77ee15f2b97c148c3cd4e
[2025-10-01 05:05:55Z INFO SelfUpdater] Which2: 'tar'
[2025-10-01 05:05:55Z INFO SelfUpdater] Location: '/usr/bin/tar'
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] Starting process:
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] File name: '/usr/bin/tar'
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] Arguments: '-xzf "/home/ubuntu/actions-runner/_work/_update/runner1.tar.gz"'
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] Working directory: '/home/ubuntu/actions-runner/_work/_update'
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] Require exit code zero: 'False'
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] Encoding web name: ; code page: ''
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] Force kill process on cancellation: 'False'
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] Redirected STDIN: 'False'
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] Persist current code page: 'False'
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] Keep redirected STDIN open: 'False'
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] High priority process: 'False'
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] Updated oom_score_adj to 500 for PID: 2594.
[2025-10-01 05:05:55Z INFO ProcessInvokerWrapper] Process started with process id 2594, waiting for process exit.
[2025-10-01 05:06:00Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished.
[2025-10-01 05:06:00Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished.
[2025-10-01 05:06:00Z INFO ProcessInvokerWrapper] Finished process 2594 with exit code 0, and elapsed time 00:00:05.7480272.
[2025-10-01 05:06:00Z INFO SelfUpdater] Finished getting latest runner package at: /home/ubuntu/actions-runner/_work/_update.
[2025-10-01 05:06:01Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:06:01Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:06:01Z INFO SelfUpdater] Copy /home/ubuntu/actions-runner/_work/_update/bin to /home/ubuntu/actions-runner/bin.2.328.0.
[2025-10-01 05:06:13Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:06:13Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:06:13Z INFO SelfUpdater] Copy /home/ubuntu/actions-runner/_work/_update/externals to /home/ubuntu/actions-runner/externals.2.328.0.
[2025-10-01 05:06:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:16:45
[2025-10-01 05:06:57Z INFO SelfUpdater] Copy any remaining .sh/.cmd files into runner root.
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:06:57Z INFO SelfUpdater] Copy /home/ubuntu/actions-runner/_work/_update/run.sh to /home/ubuntu/actions-runner/run.sh
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:06:57Z INFO SelfUpdater] Copy /home/ubuntu/actions-runner/_work/_update/config.sh to /home/ubuntu/actions-runner/config.sh
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:06:57Z INFO SelfUpdater] Copy /home/ubuntu/actions-runner/_work/_update/safe_sleep.sh to /home/ubuntu/actions-runner/safe_sleep.sh
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:06:57Z INFO SelfUpdater] Copy /home/ubuntu/actions-runner/_work/_update/run-helper.sh.template to /home/ubuntu/actions-runner/run-helper.sh.template
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:06:57Z INFO SelfUpdater] Copy /home/ubuntu/actions-runner/_work/_update/run-helper.cmd.template to /home/ubuntu/actions-runner/run-helper.cmd.template
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:06:57Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:06:57Z INFO SelfUpdater] Copy /home/ubuntu/actions-runner/_work/_update/env.sh to /home/ubuntu/actions-runner/env.sh
[2025-10-01 05:06:57Z INFO SelfUpdater] Download latest runner and unzip into runner root.
[2025-10-01 05:06:57Z INFO Terminal] WRITE LINE: Waiting for current job finish running.
[2025-10-01 05:06:57Z INFO SelfUpdater] DownloadUrl: https://github.com/actions/runner/releases/download/v2.328.0/actions-runner-linux-x64-2.328.0.tar.gz
[2025-10-01 05:06:57Z INFO SelfUpdater] PackageDownloadTime: 7562ms
[2025-10-01 05:06:57Z INFO SelfUpdater] Attempts: 1
[2025-10-01 05:06:57Z INFO SelfUpdater] PackageSize: 216MB
[2025-10-01 05:06:57Z INFO SelfUpdater] ValidateHashTime: 250ms
[2025-10-01 05:06:57Z INFO SelfUpdater] PackageExtractTime: 5753ms
[2025-10-01 05:06:57Z INFO SelfUpdater] CopyRunnerToRootTime: 56898ms
[2025-10-01 05:06:57Z INFO JobDispatcher] Waiting WorkerDispatcher for job 443720fa-89ae-5f5d-9586-66e93a8de37a run to finish.
[2025-10-01 05:07:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:17:45
[2025-10-01 05:08:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:18:45
[2025-10-01 05:09:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:19:45
[2025-10-01 05:10:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:20:45
[2025-10-01 05:11:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:21:45
[2025-10-01 05:12:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:22:45
[2025-10-01 05:13:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:23:45
[2025-10-01 05:14:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:24:45
[2025-10-01 05:15:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:25:45
[2025-10-01 05:16:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:26:45
[2025-10-01 05:17:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:27:45
[2025-10-01 05:18:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:28:45
[2025-10-01 05:19:45Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:29:45
[2025-10-01 05:20:46Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:30:46
[2025-10-01 05:21:46Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:31:46
[2025-10-01 05:22:46Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:32:46
[2025-10-01 05:23:46Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:33:46
[2025-10-01 05:24:46Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:34:46
[2025-10-01 05:25:46Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:35:46
[2025-10-01 05:26:46Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:36:46
[2025-10-01 05:27:46Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:37:46
[2025-10-01 05:28:46Z INFO JobDispatcher] Successfully renew job 443720fa-89ae-5f5d-9586-66e93a8de37a, job is valid till 10/01/2025 05:38:46
[2025-10-01 05:29:36Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished.
[2025-10-01 05:29:36Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished.
[2025-10-01 05:29:36Z INFO ProcessInvokerWrapper] Finished process 2543 with exit code 100, and elapsed time 00:23:51.0109390.
[2025-10-01 05:29:36Z INFO JobDispatcher] Worker finished for job 443720fa-89ae-5f5d-9586-66e93a8de37a. Code: 100
[2025-10-01 05:29:36Z INFO JobDispatcher] finish job request for job 443720fa-89ae-5f5d-9586-66e93a8de37a with result: Succeeded
[2025-10-01 05:29:36Z INFO Terminal] WRITE LINE: 2025-10-01 05:29:36Z: Job completed with result: Succeeded
[2025-10-01 05:29:36Z INFO JobDispatcher] Stop renew job request for job 443720fa-89ae-5f5d-9586-66e93a8de37a.
[2025-10-01 05:29:36Z INFO JobDispatcher] job renew has been cancelled, stop renew job 443720fa-89ae-5f5d-9586-66e93a8de37a.
[2025-10-01 05:29:36Z INFO JobNotification] Entering JobCompleted Notification
[2025-10-01 05:29:36Z INFO JobNotification] Entering EndMonitor
[2025-10-01 05:29:36Z INFO MessageListener] Received job status event. JobState: Online
[2025-10-01 05:29:36Z INFO JobDispatcher] Job request 443720fa-89ae-5f5d-9586-66e93a8de37a processed succeed.
[2025-10-01 05:29:36Z INFO SelfUpdater] All running job has exited.
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO SelfUpdater] Delete old version runner backup.
[2025-10-01 05:29:36Z INFO Terminal] WRITE LINE: Generate and execute update script.
[2025-10-01 05:29:36Z INFO SelfUpdater] DeleteRunnerBackupTime: 0ms
[2025-10-01 05:29:36Z ERR GitHubActionsService] PUT request to https://pipelinesghubeus21.actions.githubusercontent.com/y5XUHlnjKefN2qh2R8hmdLyJjN70qbMohF4GHkx36Nxw0ONalj/_apis/distributedtask/pools/1/agents/23952/updates?currentState=Generate%20and%20execute%20update%20script.&updateTrace=DeleteRunnerBackupTime%3A%200ms failed. HTTP Status: BadRequest
[2025-10-01 05:29:36Z ERR SelfUpdater] GitHub.DistributedTask.WebApi.TaskAgentPendingUpdateNotFoundException: No runner pending update found for runner 23952.
at GitHub.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
at GitHub.Runner.Listener.SelfUpdater.UpdateRunnerUpdateStateAsync(String currentState)
[2025-10-01 05:29:36Z INFO SelfUpdater] Catch exception during report update state, ignore this error and continue auto-update.
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Diag': '/home/ubuntu/actions-runner/_diag'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Bin': '/home/ubuntu/actions-runner/bin'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Root': '/home/ubuntu/actions-runner'
[2025-10-01 05:29:36Z INFO HostContext] Well known directory 'Work': '/home/ubuntu/actions-runner/_work'
[2025-10-01 05:29:36Z INFO SelfUpdater] Generate update script into: /home/ubuntu/actions-runner/_work/_update.sh
[2025-10-01 05:29:36Z INFO SelfUpdater] Which2: 'bash'
[2025-10-01 05:29:36Z INFO SelfUpdater] Location: '/usr/bin/bash'
[2025-10-01 05:29:36Z INFO SelfUpdater] Update script start running
[2025-10-01 05:29:36Z INFO Terminal] WRITE LINE: Runner will exit shortly for update, should be back online within 10 seconds.
[2025-10-01 05:29:36Z INFO SelfUpdater] TotalUpdateTime: 1429806ms
[2025-10-01 05:29:36Z ERR GitHubActionsService] PUT request to https://pipelinesghubeus21.actions.githubusercontent.com/y5XUHlnjKefN2qh2R8hmdLyJjN70qbMohF4GHkx36Nxw0ONalj/_apis/distributedtask/pools/1/agents/23952/updates?currentState=Runner%20will%20exit%20shortly%20for%20update%2C%20should%20be%20back%20online%20within%2010%20seconds.&updateTrace=TotalUpdateTime%3A%201429806ms failed. HTTP Status: BadRequest
[2025-10-01 05:29:36Z ERR SelfUpdater] GitHub.DistributedTask.WebApi.TaskAgentPendingUpdateNotFoundException: No runner pending update found for runner 23952.
at GitHub.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
at GitHub.Runner.Listener.SelfUpdater.UpdateRunnerUpdateStateAsync(String currentState)
[2025-10-01 05:29:36Z INFO SelfUpdater] Catch exception during report update state, ignore this error and continue auto-update.
[2025-10-01 05:29:36Z INFO Terminal] WRITE LINE: Runner update process finished.
[2025-10-01 05:29:36Z ERR GitHubActionsService] PUT request to https://pipelinesghubeus21.actions.githubusercontent.com/y5XUHlnjKefN2qh2R8hmdLyJjN70qbMohF4GHkx36Nxw0ONalj/_apis/distributedtask/pools/1/agents/23952/updates?currentState=Runner%20update%20process%20finished.&updateTrace= failed. HTTP Status: BadRequest
[2025-10-01 05:29:36Z ERR SelfUpdater] GitHub.DistributedTask.WebApi.TaskAgentPendingUpdateNotFoundException: No runner pending update found for runner 23952.
at GitHub.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
at GitHub.Runner.Listener.SelfUpdater.UpdateRunnerUpdateStateAsync(String currentState)
[2025-10-01 05:29:36Z INFO SelfUpdater] Catch exception during report update state, ignore this error and continue auto-update.
[2025-10-01 05:29:36Z INFO Runner] Auto update task finished at backend, an runner update is ready to apply exit the current runner instance.
[2025-10-01 05:29:36Z INFO Runner] Stop message queue looping.
[2025-10-01 05:29:36Z WARN GitHubActionsService] GET request to https://broker.actions.githubusercontent.com/message?sessionId=9dc7a5dd-e4fc-4552-90a7-354f9dc100cc&status=Busy&runnerVersion=2.321.0&os=Linux&architecture=X64&disableUpdate=false has been cancelled.
[2025-10-01 05:29:36Z ERR BrokerServer] Catch exception during request
[2025-10-01 05:29:36Z ERR BrokerServer] System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (125): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize)
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken)
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.AuthenticationHelper.SendWithNtAuthAsync(HttpRequestMessage request, Uri authUri, Boolean async, ICredentials credentials, Boolean isProxyAuth, HttpConnection connection, HttpConnectionPool connectionPool, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, Boolean async, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at GitHub.Services.Common.RawHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at GitHub.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Sdk.WebApi.WebApi.RawHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
at Sdk.WebApi.WebApi.RawHttpClientBase.SendAsync[T](HttpRequestMessage message, Boolean readErrorBody, Object userState, CancellationToken cancellationToken)
at Sdk.WebApi.WebApi.RawHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Uri requestUri, HttpContent content, IEnumerable`1 queryParameters, Boolean readErrorBody, Object userState, CancellationToken cancellationToken)
at GitHub.Actions.RunService.WebApi.BrokerHttpClient.GetRunnerMessageAsync(Nullable`1 sessionId, String runnerVersion, Nullable`1 status, String os, String architecture, Nullable`1 disableUpdate, CancellationToken cancellationToken)
at GitHub.Runner.Common.BrokerServer.<>c__DisplayClass7_0.<<GetRunnerMessageAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at GitHub.Runner.Common.RunnerService.RetryRequest[T](Func`1 func, CancellationToken cancellationToken, Int32 maxRetryAttemptsCount, Func`2 shouldRetry)
[2025-10-01 05:29:36Z ERR BrokerServer] #####################################################
[2025-10-01 05:29:36Z ERR BrokerServer] System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (125): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize)
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken)
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.AuthenticationHelper.SendWithNtAuthAsync(HttpRequestMessage request, Uri authUri, Boolean async, ICredentials credentials, Boolean isProxyAuth, HttpConnection connection, HttpConnectionPool connectionPool, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, Boolean async, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at GitHub.Services.Common.RawHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at GitHub.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
[2025-10-01 05:29:36Z ERR BrokerServer] #####################################################
[2025-10-01 05:29:36Z ERR BrokerServer] System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (125): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize)
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken)
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2025-10-01 05:29:36Z ERR BrokerServer] #####################################################
[2025-10-01 05:29:36Z ERR BrokerServer] System.Net.Sockets.SocketException (125): Operation canceled
[2025-10-01 05:29:36Z WARN BrokerServer] Back off 6.444 seconds before next retry. 4 attempt left.
[2025-10-01 05:29:36Z INFO MessageListener] Get next message has been cancelled.
[2025-10-01 05:29:36Z INFO Runner] Ignore any exception after cancel message loop. System.Threading.Tasks.TaskCanceledException: A task was canceled.
at GitHub.Runner.Common.RunnerService.RetryRequest[T](Func`1 func, CancellationToken cancellationToken, Int32 maxRetryAttemptsCount, Func`2 shouldRetry)
at GitHub.Runner.Listener.MessageListener.GetNextMessageAsync(CancellationToken token)
at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
[2025-10-01 05:29:36Z INFO JobDispatcher] Shutting down JobDispatcher. Make sure all WorkerDispatcher has finished.
[2025-10-01 05:29:36Z INFO Runner] Deleting Runner Session...
[2025-10-01 05:29:36Z INFO Listener] Runner execution has finished with return code 3