Problem: run task freeze

[progserega]

Issue

In web of task:

 8:30:06 AM
Task 106 added to queue
8:30:10 AM
Started: 106
8:30:10 AM
Run TaskRunner with template: node-db-05.pi.local - dev
8:30:11 AM
Preparing: 106
8:30:11 AM
Warning: Permanently added 'git.pi.local' (ED25519) to the list of known hosts.
8:30:11 AM
Updating Repository [email protected]:nic/ansible-servers-playbooks-and-roles.git
8:30:11 AM
Warning: Permanently added 'git.pi.local' (ED25519) to the list of known hosts.
8:30:11 AM
From git.pi.local:nic/ansible-servers-playbooks-and-roles
8:30:11 AM
 * branch            dev        -> FETCH_HEAD
8:30:11 AM
Already up to date.
8:30:11 AM
Get current commit hash
8:30:11 AM
Get current commit message
8:30:11 AM
Failed to update task commit

In logs:

апр 06 08:27:45 semaphore systemd[1]: Started semaphore.service - semaphore service.
апр 06 08:27:45 semaphore semaphore[116634]: Loading config
апр 06 08:27:45 semaphore semaphore[116634]: Validating config
апр 06 08:27:45 semaphore semaphore[116634]: Postgres semaphore_user@localhost semaphore
апр 06 08:27:45 semaphore semaphore[116634]: Tmp Path (projects home) /tmp/semaphore
апр 06 08:27:45 semaphore semaphore[116634]: Semaphore 2.13.0-75ffe9c-1742166024
апр 06 08:27:45 semaphore semaphore[116634]: Interface localhost
апр 06 08:27:45 semaphore semaphore[116634]: Port :3000
апр 06 08:27:45 semaphore semaphore[116634]: Server is running
апр 06 08:28:47 semaphore semaphore[116634]: time="2025-04-06T08:28:47+10:00" level=error msg="websocket: close 1006 (abnormal closure): unexpected EOF" fields.level=Error
апр 06 08:28:47 semaphore semaphore[116634]: time="2025-04-06T08:28:47+10:00" level=error msg="write tcp 127.0.0.1:3000->127.0.0.1:58022: use of closed network connection" error="Cannot send close message"
апр 06 08:30:06 semaphore semaphore[116634]: time="2025-04-06T08:30:06+10:00" level=info msg="Task 106 added to queue"
апр 06 08:30:10 semaphore semaphore[116634]: time="2025-04-06T08:30:10+10:00" level=info msg="Set resource locker with TaskRunner 106"
апр 06 08:30:10 semaphore semaphore[116634]: time="2025-04-06T08:30:10+10:00" level=info msg="Task 106 removed from queue"
апр 06 08:30:11 semaphore semaphore[116634]: time="2025-04-06T08:30:11+10:00" level=warning msg="read |0: file already closed" error="Failed to read TaskRunner output"
апр 06 08:30:11 semaphore semaphore[116634]: time="2025-04-06T08:30:11+10:00" level=warning msg="read |0: file already closed" error="Failed to read TaskRunner output"
апр 06 08:30:11 semaphore semaphore[116634]: time="2025-04-06T08:30:11+10:00" level=panic msg="pq: неверная последовательность байт для кодировки \"UTF8\": 0xd1" error="Failed to update task commit"
апр 06 08:30:11 semaphore semaphore[116634]: time="2025-04-06T08:30:11+10:00" level=error msg="remove /tmp/semaphore/inventory_106: no such file or directory"
апр 06 08:30:11 semaphore semaphore[116634]: time="2025-04-06T08:30:11+10:00" level=info msg="Stopped running TaskRunner 106"
апр 06 08:30:11 semaphore semaphore[116634]: time="2025-04-06T08:30:11+10:00" level=info msg="Release resource locker with TaskRunner 106"
апр 06 08:30:11 semaphore semaphore[116634]: time="2025-04-06T08:30:11+10:00" level=panic msg="pq: неверная последовательность байт для кодировки \"UTF8\": 0xd1" error="Failed to update TaskRunner status"
апр 06 08:30:11 semaphore semaphore[116634]: panic: (*logrus.Entry) 0xc0002f4a10
апр 06 08:30:11 semaphore semaphore[116634]:         panic: (*logrus.Entry) 0xc0002fca10
апр 06 08:30:11 semaphore semaphore[116634]: goroutine 238 [running]:
апр 06 08:30:11 semaphore semaphore[116634]: github.com/sirupsen/logrus.(*Entry).log(0xc0002fc9a0, 0x0, {0xc00008eee0, 0x69})
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:260 +0x485
апр 06 08:30:11 semaphore semaphore[116634]: github.com/sirupsen/logrus.(*Entry).Log(0xc0002fc9a0, 0x0, {0xc0001126b0?, 0xc0001126c0?, 0x469085?})
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:304 +0x48
апр 06 08:30:11 semaphore semaphore[116634]: github.com/sirupsen/logrus.(*Entry).Panic(...)
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:342
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/util.LogPanicF({0x1b35140, 0xc0002bca20}, 0xd1e551?)
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/util/errorLogging.go:39 +0x88
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).panicOnError(0x6a?, {0x1b35140, 0xc0002bca20}, {0xd3c1df, 0x22})
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/services/tasks/TaskRunner_logging.go:135 +0x165
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).saveStatus(0xc0001a0608)
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/services/tasks/TaskRunner.go:78 +0x65c
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).run.func1()
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/services/tasks/TaskRunner.go:116 +0x196
апр 06 08:30:11 semaphore semaphore[116634]: panic({0xd0e8a0?, 0xc0002f4a10?})
апр 06 08:30:11 semaphore semaphore[116634]:         /opt/hostedtoolcache/go/1.23.7/x64/src/runtime/panic.go:791 +0x132
апр 06 08:30:11 semaphore semaphore[116634]: github.com/sirupsen/logrus.(*Entry).log(0xc0002f49a0, 0x0, {0xc0002627e0, 0x69})
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:260 +0x485
апр 06 08:30:11 semaphore semaphore[116634]: github.com/sirupsen/logrus.(*Entry).Log(0xc0002f49a0, 0x0, {0xc000112e98?, 0xc000112ea8?, 0x469085?})
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:304 +0x48
апр 06 08:30:11 semaphore semaphore[116634]: github.com/sirupsen/logrus.(*Entry).Panic(...)
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/vendor/github.com/sirupsen/logrus/entry.go:342
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/util.LogPanicF({0x1b35140, 0xc000272b40}, 0xd1e551?)
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/util/errorLogging.go:39 +0x88
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).panicOnError(0x6a?, {0x1b35140, 0xc000272b40}, {0xd349ef, 0x1c})
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/services/tasks/TaskRunner_logging.go:135 +0x165
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).SetCommit(0xc0001a0608, {0xc0002840f0, 0x28}, {0xc000080500, 0x64})
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/services/tasks/TaskRunner_logging.go:72 +0x19e
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/services/tasks.(*LocalJob).SetCommit(...)
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/services/tasks/LocalJob.go:58
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/services/tasks.(*LocalJob).checkoutRepository(0xc00038ac08)
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/services/tasks/LocalJob.go:668 +0x36b
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/services/tasks.(*LocalJob).prepareRun(0xc00038ac08, {0x0, 0x0, 0x0}, {0xbd7900, 0xc0006622e8})
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/services/tasks/LocalJob.go:579 +0x2ac
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/services/tasks.(*LocalJob).Run(0xc00038ac08, {0xc0002f76c0, 0x5}, 0x0, {0x0, 0x0})
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/services/tasks/LocalJob.go:496 +0x2c7
апр 06 08:30:11 semaphore semaphore[116634]: github.com/semaphoreui/semaphore/services/tasks.(*TaskRunner).run(0xc0001a0608)
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/services/tasks/TaskRunner.go:163 +0x699
апр 06 08:30:11 semaphore semaphore[116634]: created by github.com/semaphoreui/semaphore/services/tasks.(*TaskPool).Run in goroutine 13
апр 06 08:30:11 semaphore semaphore[116634]:         /home/runner/work/semaphore/semaphore/services/tasks/TaskPool.go:186 +0x4b6
апр 06 08:30:11 semaphore systemd[1]: semaphore.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
апр 06 08:30:11 semaphore systemd[1]: semaphore.service: Failed with result 'exit-code'.
апр 06 08:30:11 semaphore systemd[1]: semaphore.service: Consumed 2.029s CPU time.
апр 06 08:30:12 semaphore systemd[1]: semaphore.service: Scheduled restart job, restart counter is at 6.
апр 06 08:30:12 semaphore systemd[1]: Stopped semaphore.service - semaphore service.
апр 06 08:30:12 semaphore systemd[1]: semaphore.service: Consumed 2.029s CPU time.
апр 06 08:30:12 semaphore systemd[1]: Started semaphore.service - semaphore service.
апр 06 08:30:12 semaphore semaphore[116756]: Loading config
апр 06 08:30:12 semaphore semaphore[116756]: Validating config
апр 06 08:30:12 semaphore semaphore[116756]: Postgres semaphore_user@localhost semaphore
апр 06 08:30:12 semaphore semaphore[116756]: Tmp Path (projects home) /tmp/semaphore
апр 06 08:30:12 semaphore semaphore[116756]: Semaphore 2.13.0-75ffe9c-1742166024
апр 06 08:30:12 semaphore semaphore[116756]: Interface localhost
апр 06 08:30:12 semaphore semaphore[116756]: Port :3000
апр 06 08:30:12 semaphore semaphore[116756]: Server is running

Impact

Service (scheduled tasks, alerts)

Installation method

Binary

Database

Postgres

Browser

Firefox

Semaphore Version

2.13.0-75ffe9c-1742166024

Ansible Version

ansible [core 2.14.18]
  config file = None
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.11.2 (main, Nov 30 2024, 21:22:50) [GCC 12.2.0] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

Logs & errors

Manual installation - system information

Linux semaphore 6.1.0-32-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.129-1 (2025-03-06) x86_64 GNU/Linux

Configuration

No response

Additional information

No response