k8sgpt icon indicating copy to clipboard operation
k8sgpt copied to clipboard

[Bug]: Trivy Analyzer has concurrent map write issue

Open michael12312 opened this issue 2 years ago • 2 comments

Checklist

  • [X] I've searched for similar issues and couldn't find anything matching
  • [X] I've included steps to reproduce the behavior

Affected Components

  • [X] K8sGPT (CLI)
  • [ ] K8sGPT Operator

K8sGPT Version

No response

Kubernetes Version

No response

Host OS and its Version

No response

Steps to reproduce

  1. Enable Trivy
  2. Install a KIND Cluster and enabled argocd
  3. Run alanlyzer

k8sgpt integrations activate trivy k8sgpt analyze

Then it failed with below error:

fatal error: concurrent map writes fatal error: concurrent map writes

goroutine 48 [running]:

k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypeWithName(0xc00056dd50, {{0x10a763d15, 0x16}, {0x10a7379e5, 0x8}, {0x10aff49c3, 0x1c}}, {0x10b91fcd0, 0xc0005741c0}) /Users/test/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/scheme.go:174 +0x2eb k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypes(0xc00056dd50, {{0x10a763d15, 0x16}, {0x10a7379e5, 0x8}}, {0xc000eb3b40, 0x16, 0x16}) /Users/test/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/scheme.go:148 +0x2af github.com/aquasecurity/trivy-operator/pkg/apis/aquasecurity/v1alpha1.addKnownTypes(0xc00056dd50) /Users/test/go/pkg/mod/github.com/aquasecurity/[email protected]/pkg/apis/aquasecurity/v1alpha1/register.go:22 +0x12b0 k8s.io/apimachinery/pkg/runtime.(*SchemeBuilder).AddToScheme(0x10d32c040, 0xc00056dd50) /Users/test/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/scheme_builder.go:29 +0x82 github.com/k8sgpt-ai/k8sgpt/pkg/integration/trivy.TrivyAnalyzer.analyzeVulnerabilityReports({0x1, 0x0}, {0xc00087d440, {0x10b9365d0, 0x10d3b4d88}, {0x0, 0x0}, {0x0, 0x0}, 0x0, ...}) /Users/test/A10/k8sgpt/pkg/integration/trivy/analyzer.go:43 +0x102 github.com/k8sgpt-ai/k8sgpt/pkg/integration/trivy.TrivyAnalyzer.Analyze({0x1, 0x0}, {0xc00087d440, {0x10b9365d0, 0x10d3b4d88}, {0x0, 0x0}, {0x0, 0x0}, 0x0, ...}) /Users/test/A10/k8sgpt/pkg/integration/trivy/analyzer.go:161 +0xbe github.com/k8sgpt-ai/k8sgpt/pkg/analysis.(*Analysis).RunAnalysis.func3({0x10b903e80, 0xc000d2721a}, {0xc001494060, 0x13})

/Users/test/A10/k8sgpt/pkg/analysis/analysis.go:271 +0x155

created by github.com/k8sgpt-ai/k8sgpt/pkg/analysis.(*Analysis).RunAnalysis in goroutine 1 /Users/test/A10/k8sgpt/pkg/analysis/analysis.go:269 +0xb74

Expected behaviour

no concurrent map write fatal error.

Actual behaviour

fatal error

Additional Information

No response

michael12312 avatar Jul 19 '24 06:07 michael12312

hi, I will fix this, and create PR soon

michael12312 avatar Jul 19 '24 06:07 michael12312

I have implemented a solution and I'm ready to submit a PR if you approve.

  1. Proposed Solution: To address this issue, we propose introducing a mutex lock to synchronize access to the mergedAnalyzerMap. This will ensure that only one goroutine can modify the map at a time, preventing race conditions and maintaining data integrity.

  2. Implementation Steps: a. Import the sync package in the relevant files. b. Add a sync.Mutex field to the Trivy struct in the trivy.go file. c. Modify the AddAnalyzer method to use the mutex for synchronization:

    • Acquire the lock before modifying the mergedAnalyzerMap.
    • Release the lock after the modification is complete. d. Update any other methods that access or modify the mergedAnalyzerMap to use the mutex.
  3. Impact Analysis:

    • Positive impacts:
      • Eliminates race conditions and ensures data integrity.
      • Improves overall stability and reliability of the Trivy Analyzer integration.
    • Potential drawbacks:
      • Slight performance overhead due to mutex operations.
      • May introduce potential deadlocks if not implemented carefully.
  4. Testing Plan: a. Unit Tests:

    • Develop unit tests that simulate concurrent access to the AddAnalyzer method.
    • Verify that the mutex correctly synchronizes access to the mergedAnalyzerMap. b. Integration Tests:
    • Create integration tests that exercise the Trivy Analyzer in a concurrent environment.
    • Ensure that no race conditions occur during parallel execution of multiple analyzers. c. Performance Testing:
    • Conduct performance tests to measure the impact of the mutex on overall system performance.
    • Compare results with the previous implementation to ensure acceptable performance. d. Code Review:
    • Perform a thorough code review to ensure proper implementation of the mutex and identify any potential issues.
  5. Implementation Details: File: /home/ubuntu/k8sgpt/pkg/integration/trivy/trivy.go

    • Add a new field to the Trivy struct: mu sync.Mutex
    • Modify the AddAnalyzer method:
      func (t *Trivy) AddAnalyzer(mergedMap *map[string]common.IAnalyzer) {
          t.mu.Lock()
          defer t.mu.Unlock()
          // Existing code for adding analyzers
      }
      

    File: /home/ubuntu/k8sgpt/pkg/analyzer/analyzer.go

    • Ensure that the call to AddAnalyzer is thread-safe:
      in.AddAnalyzer(&mergedAnalyzerMap)
      
      (This call is already correct, as the synchronization is handled within the AddAnalyzer method)

rohitg00 avatar Aug 08 '24 10:08 rohitg00