bug(scan): scan fails to create new OS thread

[nierz]

Hi,

It seems like KICS fails to scan a large repository (about 9000 files). The reason seems to be unlimited concurrent workers in the Preparing Scan Assets stage.

Assuming there's no way to limit the amount of workers spawned - here's the function that goes crazy in spawning workers: pkg/analyzer/analyzer.go:Analyze More specifically, this part in the code:

	// Start the workers
	for _, file := range files {
		wg.Add(1)
		// analyze the files concurrently
		a := &analyzerInfo{
			typesFlag:        a.Types,
			excludeTypesFlag: a.ExcludeTypes,
			filePath:         file,
		}
		go a.worker(results, unwanted, locCount, &wg)
	}

Our use-case is quite unique which is probably why this hasn't been a problem so far: We're trying to scan a remote repository on a network mount, and I'm assuming the slower responsiveness makes the workers spawn more quickly than the previous ones are able to finish their job. Even though this use-case isn't the obvious one, I still think it's a problem for this part to be unlimited (assuming I didn't just miss how to limit it).

The other constraining part is that we're running this scan on a Kubernetes pod, which is limited to 1000 PIDs.

Expected Behavior

Limit the amount of workers - if not as a customizable variable then at least so KICS doesn't crash

Actual Behavior

Scan fails due to failing to create a new thread. Here's the log:

Preparing Scan Assets:  \runtime: failed to create new OS thread (have 989 already; errno=11)
runtime: may need to increase max user processes (ulimit -u)
fatal error: newosproc 

Steps to Reproduce the Problem

kics scan -p .

Run on a really big repository or a slow network drive. Alternatively, artificially limit the maximum number of threads.

Specifications

(N/A if not applicable)

• Version: v1.7.12
• Platform: Kubernetes
• Subsystem: N/A