Bug? EXC_BAD_ACCESS (code=1, address=0x10) runtime error

[wirhabenzeit]

Hi, I am using SwiftSlash within a task group

try await withThrowingTaskGroup(of: Void.self) { group 
  for cmd in cmds {
    group.addTask {
      let commandResult = try await Command(bash:cmd).runSync()
        guard commandResult.exitCode == 0 else {
          throw MyErrors.someError
        }
    }
  }
  try await group.waitForAll()
}

but sometimes get an error of the type Thread 2: EXC_BAD_ACCESS (code=1, address=0x10). While the thread varies, the problematic address seems to be the same for every runtime error.

The error seems to originate from

_ = self.writerTrigger.updateValue(newOut.continuation, forKey:newPipe.writing)

on line 232 of ProcessSpawner.swift.

Is there anything I can do to prevent the access of the same address for multiple threads?

[tannerdsilva]

Hey there @wirhabenzeit, thanks for using SwiftSlash and reaching out about your issue!

This is certainly a curious issue - I've done rigorous testing on this framework over the last year and can't remember anything exactly like this.

SwiftSlash has three primary unit tests that all run in highly concurrent TaskGroups, similar to your sample code. This is certainly a design pattern I want SwiftSlash to be able to facilitate for developers.

The particular line of code you called out (232 in ProcessSpawner.swift) is an actor-isolated context. Furthermore, I have a very high level of confidence in the thread safety of the framework (in current release), so we will need to dive in to explain what we're seeing on the surface here.

I assume you're using version 3.3.1 since it has been stable for almost a year.

I have two questions for you:

• Can you please provide basic info about your system? RAM capacity and operating system primarily. CPU count would be helpful too.

• Can you please help me understand what these commands are doing? Do they run for a long time? Do the commands produce a lot of output?

My initial theory: I suspect you're hitting some kind of system limit (maybe memory related). The most common type of system-related limit SwiftSlash encounters is a file-descriptor limit (per process), however, it appears your particular workload has found some new extreme use-case I'd like to design for.

Is there anything I can do to prevent the access of the same address for multiple threads?

As stated above, I have no reason to believe the current release is actually unsanitary with its thread safety. If my current theory holds true, you should be able to restore stability to your application if you simply limit the number of concurrent shell commands to be no more than the CPU count of your machine. (Example: run no more than 6 concurrent commands if on a 6 core system - as opposed to launching all the commands immediately). This is a conservative number, but tends to be a good "rule of thumb" for any given workload/system.

[tannerdsilva]

4 weeks with no response here - switching posture from a "supporting" role into a "directing" role.

I am confident this is a straightforward resource-exhaustion situation. You are (likely) launching too many commands at once, resulting in memory exhaustion or file-handle exhaustion. These limits are configurable, but by default, defined by your system.

Solution: rate-limit your commands. The safest rule-of-thumb is to launch no more processes than your system has CPU cores (example: 4 cores = 4 concurrent shell commands).

Else, you should adjust your system resources to accommodate the concurrency you are building in your application.

[tannerdsilva]

This issue is not related to a systematic flaw with the codebase. Rather, this issue relates to a lack of system resources for the process you are trying to build.

As such: this issue will be closed with the upcoming 4.0.0 release if no further information is provided.