fatal error: concurrent map read and map write

[cowley05]

Describe the bug We are seeing intermittent errors when running talisman --scan.

We use talisman as a cli mainly and have not seen the same errors occur when using as a pre-commit hook installed in a single repo.

We see the message "fatal error: concurrent map read and map write" followed by lots of logs from many places in the talisman code. (see snippet below, had to truncate as way too long but added a file to show whole output).

To Reproduce Steps to reproduce the behavior:

1. Install talisman globally; we have seen this using the install globally method in the readme, brew installed version and using the binary from GH releases
2. Add exceptions such as:

fileignoreconfig:
- filename: somepath/somefile.clj
  allowed_patterns: [some-pattern]
- filename: somepath/*.sql
  ignore_detectors: [filename]
allowed_patterns:
- some-pattern
- some-other-pattern

3. run talisman --scan
4. See error below, it is flaky so sometimes it will appear and sometimes it will run fine

Expected behavior I expect an error not to occur

Screenshots

Talisman Scan: 534 / 1710 <------------------------------------------------------------↖......................................................................................................................................> 31.23%  
fatal error: concurrent map read and map write

goroutine 1850 [running]:
runtime.throw(0x13c0ca3, 0x21)
        runtime/panic.go:1116 +0x72 fp=0xc001a80c68 sp=0xc001a80c38 pc=0x1036c72
runtime.mapaccess2_faststr(0x1362be0, 0xc0001b18f0, 0xc000afc8c0, 0x17, 0xc001a80d28, 0x105198f)
        runtime/map_faststr.go:116 +0x4a5 fp=0xc001a80cd8 sp=0xc001a80c68 pc=0x1015c65
talisman/detector/pattern.replaceAllStrings(0xc000cbea00, 0x1a3e, 0xc000afc8c0, 0x17, 0x1a3e, 0x1a80)
        talisman/detector/pattern/pattern_detector.go:86 +0x51 fp=0xc001a80d38 sp=0xc001a80cd8 pc=0x12f8331
talisman/detector/pattern.processAllowedPatterns(0xc00001ee69, 0x2a, 0xc00001ee80, 0x13, 0xc00059f8e0, 0x2, 0x2, 0xc000cbcf80, 0x1a3e, 0x1a80, ...)
        talisman/detector/pattern/pattern_detector.go:97 +0x311 fp=0xc001a80e68 sp=0xc001a80d38 pc=0x12f8791
talisman/detector/pattern.PatternDetector.Test.func1(0xc001288870, 0xc0004aaca0, 0xc000af6120, 0xc000f55b00, 0xc0004f3440, 0xc0005938c0, 0xc0004f3380, 0xc00001ee69, 0x2a, 0xc00001ee80, ...)
        talisman/detector/pattern/pattern_detector.go:53 +0x16d fp=0xc001a80f58 sp=0xc001a80e68 pc=0x12f954d
runtime.goexit()
        runtime/asm_amd64.s:1374 +0x1 fp=0xc001a80f60 sp=0xc001a80f58 pc=0x106b001
created by talisman/detector/pattern.PatternDetector.Test
        talisman/detector/pattern/pattern_detector.go:46 +0x238

goroutine 1 [select]:
talisman/detector/pattern.PatternDetector.Test(0xc0005938c0, 0x1681d60, 0xc000b881e0, 0x1680660, 0x18d6e30, 0xc000af6120, 0xc000808000, 0x23a, 0x266, 0xc000af6120, ...)
        talisman/detector/pattern/pattern_detector.go:63 +0x3b1
talisman/detector.(*Chain).Test(0xc000593800, 0xc000808000, 0x23a, 0x266, 0xc000af6120, 0xc000021980)
        talisman/detector/chain.go:61 +0x476
main.(*Runner).Scan(0xc000f3bd78, 0x0, 0x0, 0x18d6e00, 0x0)
        talisman/runner.go:58 +0x125
main.run(0x167fe00, 0xc000010010, 0x0, 0x13b5aaf, 0x8, 0x0, 0x0, 0x1, 0x0, 0x0, ...)
        talisman/talisman.go:129 +0x80a
main.main()
        talisman/talisman.go:104 +0x5b4

goroutine 2242 [runnable]:
talisman/detector/pattern.PatternDetector.Test.func1(0xc001288870, 0xc0004aaca0, 0xc000af6120, 0xc000f55b00, 0xc0004f3440, 0xc0005938c0, 0xc0004f3380, 0xc00072f169, 0x2a, 0xc00072f180, ...)
        talisman/detector/pattern/pattern_detector.go:46
created by talisman/detector/pattern.PatternDetector.Test
        talisman/detector/pattern/pattern_detector.go:46 +0x238

Desktop (please complete the following information):

• OS: Mac OS/Alpine Linux docker container

Example output output.txt

[cowley05]

Anyone care to comment?

[harinee]

Hey, thanks for your patience. Tough times for most of the contributors here with the recent Covid wave. We are getting back :) Could this be related to https://github.com/thoughtworks/talisman/issues/301 ? Is your issue reproducible on trees of larger sizes too?

[svishwanath-tw]

@cowley05 : Is this still an issue with release 1.27.0 ?