copilot-cli icon indicating copy to clipboard operation
copilot-cli copied to clipboard

Bash Tool Fails with posix_spawnp Error After Extended Use

Open hamadycisselp360 opened this issue 4 months ago โ€ข 1 comments

Describe the bug

Running bash tools consistently fails after a period of time with the error: <exited with error: posix_spawnp failed.>

Affected version

0.0.358

Steps to reproduce the behavior

 1. Start multiple bash operations or long-running bash commands
 2. After some time (varies), bash tool execution fails
 3. Error appears to be related to process spawning

Expected behavior

Bash tools should execute reliably without timing out or failing due to process spawning issues.

Additional context

Error Details

 - **Error Type**: posix_spawnp failure
 - **Component**: Bash tool
 - **Frequency**: Intermittent, occurs after extended usage
 
 ## Environment
 - OS: macOS/Linux (POSIX-based)
 - Bash Tool Version: [Current]
 - Date Reported: 2025-11-27
 
 ## Possible Root Causes
 - File descriptor exhaustion
 - Process limit reached
 - Resource cleanup not happening properly
 - Signal handling issues
 
 ## Suggested Investigation
 - Check system resource limits (ulimit)
 - Verify proper cleanup of spawned processes
 - Review signal handling and process termination
 - Investigate file descriptor management
 
 ## Related Issues
 - None identified yet

hamadycisselp360 avatar Nov 27 '25 14:11 hamadycisselp360

I am seeing this error also, macOS Tahoe 26.1 and using Ghostty terminal:

copilot --version 0.0.365 Commit: 76d0881

When it happens starting a new terminal tab in Ghostty also kind of goes bonkers and fails to open a new tab.

sj4nes avatar Dec 01 '25 06:12 sj4nes

I rebooted macOS completely and the issue hasn't reappeared so far ๐Ÿคž

sj4nes avatar Dec 02 '25 18:12 sj4nes

I had already rebooted few days ago (Friday), I have started working with Copilot CLI since yesterday and hadn't seen the issue yet. I will monitor and eventually close the issue if it does not come back,

thanks

hamadycisselp360 avatar Dec 02 '25 20:12 hamadycisselp360

I knew I was jinxing myself. That error has returned after a couple more hours.

sj4nes avatar Dec 02 '25 20:12 sj4nes

I switched from Ghostty to Kitty and copilot is able to spawn commands-- just for a bit. The spawnp error returned quickly after that, so my gut feeling is this is a macOS issue in general. I started a clean session again and watched PTY usage:

sjanes@MBP-LHYP9PW370 accession-console % while true; do sleep 3; lsof | grep -E "ttys|pty" | wc -l; done
     118
     123
     123
     123
     123
     123
     123
     138
     138
     138
     143
     143
     143
     143
     143
     143
     143
     148
     153
     163
     173
     183
     198
     198
     208
     218
     218
     218
     218
     218
     218
     218
     218

When it got to 218 in the system the spawnp error appeared. When I cancelled the agent's activity before it started thrashing on trying different things to execute (it had two spawnp errors in fast succession) the PTY level dropped to 68. So that was interesting. The PTY use didn't decrease until I ESC-cancelled the agent's activity. In the same session I told the agent to resume with the more complicated pipeline it first attempted and it was able to execute that without the error.

sj4nes avatar Dec 02 '25 21:12 sj4nes

Seeing the same issue running copilot CLI with batch jobs. Mac OS X Tahoe running Visual Studio Code terminal running zsh running Copilot CLI running subagent prompt.

mozumder avatar Dec 07 '25 23:12 mozumder

[!IMPORTANT] I have a personal stake in obtaining the spent premium points. This would be cordial.

I hope this helps the developers and project stakeholders to find and resolve the issue:


I can give as feedback that it looks like that copilot CLI is just opening new shell sessions and that we reach the limit with that behavior quickly. I explained to Copilot that he should investigate why we get this spawn issue and what he should do when he runs into that again and try to analyze and document it thenโ€ฆ

What happened is this:

Image

Here is the start of my prompt:

 > Can you continue with the work, not all tasks from the original prompt are yet done. If you get again the issues with the terminal, can you try to close a bahs session and check if the issue perist, and if not can you then debug if the shell / bash issue is based on how many processes or so are open on my macos? if the issue is related by `ulimit -a`, tty, pty amount, etc.? That happens oftener and it is something what is related to the amount of processes or forks we or you open.
Image Image Image

It looks like when copilot CLI runs, it doesn't clean up its forked processes?

Image

When I exit the copilot session, I can then open a new tab in iterm or Terminal...

 > I closed the Copilot session and now you should be able to continue to work and you might can analyse the terminal / spawn
   issue, so that I can report it to Github. I have the feeling that when you open many bash-sessions / tools that they are not
   properly cleaned up as long your parent process is running?

 โœ” List shells
   โ”” <no active sessions>

 โ— Good, sessions are clean now. Let me first check GitHub forks status and then continue with creating the documentation:

 โœ” Check existing forks in GitHub account
    $ gh repo list tobiashochguertel --fork --json nameWithOwner,name 2>/dev/null | head -50
    โ”” 2 lines...

 โœ” List directory usage-lib-src
   โ”” 37 files found

 โœ” Check mise repo exists
    $ ls -la /Users/tobiashochgurtel/mise 2>/dev/null | head -20
    โ”” 21 lines...

 โ— Both forks exist (tobiashochguertel/usage and tobiashochguertel/mise). Now let me verify the usage-lib-src repository is your
   fork and create the documentation structure:

 โœ” Check usage-lib-src remotes
    $ cd /Users/tobiashochgurtel/usage-lib-src && git remote -v
    โ”” 5 lines...

 โœ” Check mise remotes
    $ cd /Users/tobiashochgurtel/mise && git remote -v
    โ”” 7 lines...

 โ— Both repositories are set up with forks. Now let me examine the code to verify the terminal width issue and document everything
   properly:

 โœ” Read usage-lib-src/lib/src/docs/layout.rs
   โ”” 255 lines read

 โœ” Check existing branches in usage repo
    $ cd /Users/tobiashochgurtel/usage-lib-src && git branch -a | head -20
    โ”” 16 lines...

 โœ” Check existing branches in mise repo
    $ cd /Users/tobiashochgurtel/mise && git branch -a | head -20
    โ”” 21 lines...

 โ— Excellent! I can see that:

     - The usage-lib-src repo already has the fix branch fix/terminal-width-wrapping
     - The code has been modified with the proper terminal width detection

   Now let me check the Cargo.toml to verify the terminal_size dependency was added, then create the documentation:

 โœ” Check terminal_size dependency
    $ cd /Users/tobiashochgurtel/usage-lib-src && grep -n "terminal_size" lib/Cargo.toml
    โ”” 2 lines...

 โœ” Create documentation directory
    $ mkdir -p /Users/tobiashochgurtel/docs/research/mise-improvements
    โ”” 1 line...

 โ— The dependency is added. Now let me research the task description issue in the mise codebase:

 โœ” Grep "description" in *.rs (/Users/tobiashochgurtel/mise/src/cli/tasks)
   โ”” 3 files found

 โœ” Grep "task.*help|help.*task" in *.rs (/Users/tobiashochgurtel/mise/src)
   โ”” 8 files found

 โœ” Grep "SpecUsage|usage::Spec" in *.rs (/Users/tobiashochgurtel/mise/src)
   โ”” 4 files found

 โœ” Read mise/src/task/mod.rs:1-100
   โ”” 105 lines read

 โœ” Read mise/src/task/task_script_parser.rs:1-100
   โ”” 105 lines read

 โœ” Grep "to_usage_spec|SpecUsage" in *.rs (/Users/tobiashochgurtel/mise/src)
   โ”” No matches found

 โœ” Grep "usage::Spec" in *.rs (/Users/tobiashochgurtel/mise/src)
   โ”” 261 lines found

 โœ” List shells
   โ”” Active shell sessions:

 โ— You're correct. Looking at the list_bash output, I can see 8 bash sessions (sessionId 0-7) that are still "active" even though
   they've all completed (status: completed).

   Key observations for your bug report:

     - Sessions persist after completion - All sessions show status: completed (exit code: 0) but remain in the active sessions
   list
     - No automatic cleanup - Sessions from ~54 seconds ago (0, 1) through ~28 seconds ago (6, 7) are all still tracked
     - Resource accumulation - Each session has a PID, suggesting actual processes or resources being held
     - Unread output retained - Most sessions still have unread output: yes, meaning buffers aren't released

   Bug hypothesis: The Copilot CLI's bash session manager:

     - Creates new sessions for parallel tool calls
     - Marks them "completed" but doesn't terminate/cleanup
     - Accumulates sessions until... (the crash you experienced?)

   For your GitHub issue, I'd suggest:

     ## Issue: Bash sessions not cleaned up after completion

     **Observed behavior:**
     - `list_bash` shows 8+ sessions with `status: completed` but still "active"
     - Each has a PID and retains unread output buffers
     - Sessions accumulate during a session lifetime

     **Expected behavior:**
     - Completed sync sessions should be cleaned up automatically
     - Resources (PIDs, buffers) should be released

     **Impact:**
     - Potential resource exhaustion during long sessions
     - May contribute to PTY/terminal instability

   Want me to check if I can manually stop these sessions, or would you prefer I leave them as evidence for debugging?

 > Can you try to stop the sessions with your builtin tools or functions and if that not works can you try to close them manually? And can you
   debug the issue somehow further and document it for Github? You might want to stop the sessions then after the debugging?

 โœ” List shells
   โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
   โ”‚ <no active sessions> โ”‚
   โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ

 โ— Good - no active sessions currently. Let me check system resources and then proceed with the work:

 โœ” Check system resources for session debugging
    $ echo "=== System Resource Check ===" && echo "PTY devices:" && ls /dev/pty* 2>/dev/null | wc -l && echo "TTY devices:" && ls /dev/tty*
    2>/dev/null | wc -l && echo "" && echo "=== ulimit -a ===" && ulimit -a && echo "" && echo "=== Process count for current user ===" && ps aux |
    grep -c "$(whoami)" && echo "" && echo "=== Open file descriptors (approximate) ===" && lsof 2>/dev/null | wc -l
   โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
   โ”‚ === System Resource Check ===                                                                                                                 โ”‚
   โ”‚ PTY devices:                                                                                                                                  โ”‚
   โ”‚      128                                                                                                                                      โ”‚
   โ”‚ TTY devices:                                                                                                                                  โ”‚
   โ”‚      531                                                                                                                                      โ”‚
   โ”‚                                                                                                                                               โ”‚
   โ”‚ === ulimit -a ===                                                                                                                             โ”‚
   โ”‚ core file size              (blocks, -c) 0                                                                                                    โ”‚
   โ”‚ data seg size               (kbytes, -d) unlimited                                                                                            โ”‚
   โ”‚ file size                   (blocks, -f) unlimited                                                                                            โ”‚
   โ”‚ max locked memory           (kbytes, -l) unlimited                                                                                            โ”‚
   โ”‚ max memory size             (kbytes, -m) unlimited                                                                                            โ”‚
   โ”‚ open files                          (-n) 1048575                                                                                              โ”‚
   โ”‚ pipe size                (512 bytes, -p) 1                                                                                                    โ”‚
   โ”‚ stack size                  (kbytes, -s) 8176                                                                                                 โ”‚
   โ”‚ cpu time                   (seconds, -t) unlimited                                                                                            โ”‚
   โ”‚ max user processes                  (-u) 10666                                                                                                โ”‚
   โ”‚ virtual memory              (kbytes, -v) unlimited                                                                                            โ”‚
   โ”‚                                                                                                                                               โ”‚
   โ”‚ === Process count for current user ===                                                                                                        โ”‚
   โ”‚ 638                                                                                                                                           โ”‚
   โ”‚                                                                                                                                               โ”‚
   โ”‚ === Open file descriptors (approximate) ===                                                                                                   โ”‚
   โ”‚    29394                                                                                                                                      โ”‚
   โ”‚ <exited with exit code 0>                                                                                                                     โ”‚
   โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
โ— Done! Here's the summary:

   Session Cleanup Results

   Found & Stopped: 6 stale sessions (IDs 8-13)

     - All were "completed" but still persisted in memory
     - Most had "unread output: yes"

   Key Findings

     - Completed sessions don't auto-cleanup - They persist until explicitly stopped
     - stop_bash works - Successfully terminates even finished sessions
     - No sessions remaining - Verified with list_bash

   Documentation Created

   ๐Ÿ“„ /Users/tobiashochgurtel/docs/mise-improvements/99-copilot-cli-session-debugging.md

   Contains:

     - Issue symptoms and debugging procedures
     - System resource checks
     - Session management tools reference
     - Findings from this debug session
     - GitHub issue template ready to file

   Would you like me to view the complete documentation or help file a GitHub issue?

I attached the created documentation: 99-copilot-cli-session-debugging.md

99-copilot-cli-session-debugging.md

tobiashochguertel avatar Dec 10 '25 00:12 tobiashochguertel

Workaround: kill the session and copilot --resume

lukaszsamson avatar Dec 20 '25 07:12 lukaszsamson

This happens all the time and is very tedious to manage especially when running multiple sessions

crmitchelmore avatar Jan 13 '26 22:01 crmitchelmore

Workaround: kill the session and copilot --resume

Thanks ๐Ÿ‘

bbloch-veepee avatar Jan 14 '26 10:01 bbloch-veepee