ggshield icon indicating copy to clipboard operation
ggshield copied to clipboard

Published 20 hours ago • verified publisher icon GitGuardian

SSLEOFError when scanning a windows docker image

Open testworksau opened this issue 1 year ago • 42 comments

Environment

  • ggshield version: 1.16.0
  • Operating system (Linux, macOS, Windows): Windows
  • Operating system version: Server 2019
  • Python version: 3.11

Describe the bug

We are seeing this issue on one of our builds:

ERROR: Error scanning: HTTPSConnectionPool(host='api.gitguardian.com', port=443): Max retries exceeded with url: /v1/multiscan?ignore_known_secrets=True (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))

Steps to reproduce:

  1. Run ggshield secret scan docker label:tag --docker-timeout=1200

Actual result:

ERROR: Error scanning: HTTPSConnectionPool(host='api.gitguardian.com', port=443): Max retries exceeded with url: /v1/multiscan?ignore_known_secrets=True (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))

Expected result:

The build should work successfully.

testworksau avatar Jun 06 '23 05:06 testworksau

Hi, how often does this happen? Does it always happen with the same Docker image?

agateau-gg avatar Jun 07 '23 15:06 agateau-gg

Hi @agateau-gg 👋🏼

I tried this twice in a row - the second not too long after the first one failed, to see if it was transient or not.

testworksau avatar Jun 13 '23 07:06 testworksau

I've just tried again this morning, and have the same issue @agateau-gg

testworksau avatar Jun 14 '23 00:06 testworksau

Sorry for the late reply.

Is your CI running behind an HTTP proxy? If so, what you are experiencing could be urllib3 issue #2164, which is in turn caused by Python issue #86793. This bug has been fixed in Python 3.9.13, 3.10.5 and in 3.11.0.

agateau-gg avatar Jun 20 '23 13:06 agateau-gg

Hi @agateau-gg

We are running Python 3.11.4; at the time I raised the issue, we would have been running 3.11.3

Python reports no proxies are in use either:

image

testworksau avatar Jun 21 '23 00:06 testworksau

Having more info could help us debug that. Can you try to add a call to ggshield --debug api-status before the call to ggshield secret scan docker?

You can also try adding --allow-self-signed to ggshield secret scan docker. If the problem goes away then it's quite possible there is an issue with your SSL setup on the machine or in the way it accesses our servers. I would not recommend leaving the option indefinitely though, since it weakens security.

agateau-gg avatar Jun 21 '23 10:06 agateau-gg

Here's the output of the debug command, and the secret scan command with --allow-self-signed enabled:

2023-06-29 04:15:38,660 DEBUG 1964:23cc ggshield.cmd.debug_logs:setup_debug_logs:44 args=['C:\\docker-image-builder\\ggshield\\Scripts\\ggshield', '--debug', 'api-status']
2023-06-29 04:15:38,660 DEBUG 1964:23cc ggshield.cmd.debug_logs:setup_debug_logs:45 py-gitguardian=1.8.0
2023-06-29 04:15:38,668 DEBUG 1964:23cc ggshield.core.config.user_config:load:144 No global config
2023-06-29 04:15:38,668 DEBUG 1964:23cc ggshield.core.config.user_config:load:153 No local config
2023-06-29 04:15:38,668 DEBUG 1964:23cc ggshield.core.git_shell:git:80 command=['rev-parse', '--show-toplevel']
2023-06-29 04:15:38,670 DEBUG 1964:23cc ggshield.core.git_shell:_get_git_path:44 Found git at C:\Program Files\Git\mingw64\bin\git.EXE
2023-06-29 04:15:38,754 DEBUG 1964:23cc ggshield.core.git_shell:git:80 command=['rev-parse', '--show-toplevel']
2023-06-29 04:15:38,781 DEBUG 1964:23cc ggshield.core.config.config:api_key:149 Using API key from $GITGUARDIAN_API_KEY
2023-06-29 04:15:38,794 DEBUG 1964:23cc urllib3.connectionpool:_new_conn:1014 Starting new HTTPS connection (1): api.gitguardian.com:443
2023-06-29 04:15:39,286 DEBUG 1964:23cc urllib3.connectionpool:_make_request:473 https://api.gitguardian.com:443 "GET /v1/health HTTP/1.1" 200 27
2023-06-29 04:15:39,301 DEBUG 1964:23cc pygitguardian.client:request:204 method=get endpoint=health status_code=200 duration=0.517515
API URL: https://api.gitguardian.com
Status: healthy
App version: v2.33.0
Secrets engine version: 2.92.0

2023-06-29 04:15:39,302 DEBUG 1964:23cc ggshield.core.check_updates:check_for_updates:60 Checking the latest released version of ggshield...
2023-06-29 04:15:39,306 DEBUG 1964:23cc urllib3.connectionpool:_new_conn:1014 Starting new HTTPS connection (1): api.github.com:443
2023-06-29 04:15:39,628 DEBUG 1964:23cc urllib3.connectionpool:_make_request:473 https://api.github.com:443 "GET /repos/GitGuardian/GGShield/releases/latest HTTP/1.1" 200 975
ggshield secret scan docker artifactory.our.domain/docker/our-dotnet-framework:1.0 --docker-timeout=1200 --allow-self-signed
Saving docker image... OK

...

Skipped
\Files\Windows\servicing\Packages\Package_7_for_KB5027131~31bf3856ad364e35~amd6
4~~10.0.4050.2.cat: can't detect encoding
Skipped \Hives\DefaultUser_Delta: can't detect encoding
Skipped \Hives\Sam_Delta: can't detect encoding
Skipped \Hives\Security_Delta: can't detect encoding
Skipped \Hives\Software_Delta: can't detect encoding
Skipped \Hives\System_Delta: can't detect encoding
Scanning... -------------------      85% 2227 files scanned out of 2630 0:00:02
 
Error: Scanning failed: HTTPSConnectionPool(host='api.gitguardian.com', port=443): Max retries exceeded with url: /v1/multiscan?ignore_known_secrets=True (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))
WARNING: GitGuardian Shield returned non-zero exit code

testworksau avatar Jun 29 '23 04:06 testworksau

The SSL errors may be caused by multiple network requests stepping on each other.

There is an undocumented environment variable to limit the number of network requests: GG_MAX_WORKERS. Unfortunately it turns out secret scan docker does not honor this variable. I just pushed a branch called agateau/docker-max-workers to fix that.

Can you try the following?

  • Install ggshield from the agateau/docker-max-workers branch (can be done with pip install --force-reinstall git+https://github.com/gitguardian/ggshield@agateau/docker-max-workers),
  • set GG_MAX_WORKERS to 1,
  • run a scan and see if you still hit the problem.

You can verify the environment variable is taken into account by running the command with --debug. There should be a log line like this (note the "scan_threads=1" at the end):

2023-06-29 17:13:10,919 DEBUG 1c71e:7f4f8f97b000 ggshield.secret.secret_scanner:scan:102
  files=<ggshield.secret.secret_scanner.SecretScanner object at 0x7f4f8ccf9780>
  command_id=7125d549-65ed-4352-b07f-0229a4beaf1e scan_threads=1

agateau-gg avatar Jun 29 '23 15:06 agateau-gg

Thanks - I have set the env var via:

$Env:GG_MAX_WORKERS = "1"

which should work as that's the same syntax we are using elsewhere in our powershell script.

I tried again with the branch build but still have the same problem. I can't see the scan_threads output so I'm not sure at this stage as to whether ggshield has detected it:

Collecting git+https://github.com/gitguardian/ggshield@agateau/docker-max-workers
Cloning https://github.com/gitguardian/ggshield (to revision agateau/docker-max-workers) to z:\temp\pip-req-build-_5evamal
Running command git clone --filter=blob:none --quiet https://github.com/gitguardian/ggshield 'Z:\Temp\pip-req-build-_5evamal'
Running command git checkout -b agateau/docker-max-workers --track origin/agateau/docker-max-workers
branch 'agateau/docker-max-workers' set up to track 'origin/agateau/docker-max-workers'.
Switched to a new branch 'agateau/docker-max-workers'
Resolved https://github.com/gitguardian/ggshield to commit c1e0e2eed827db362acd42de7feafd9ce89ef91e
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
...
2023-06-30 05:42:34,984 DEBUG 1274:152c ggshield.cmd.debug_logs:setup_debug_logs:44 args=['Z:\\\\docker-image-builder\\ggshield\\Scripts\\ggshield', '--debug', 'api-status']
2023-06-30 05:42:34,984 DEBUG 1274:152c ggshield.cmd.debug_logs:setup_debug_logs:45 py-gitguardian=1.8.0
2023-06-30 05:42:34,987 DEBUG 1274:152c ggshield.core.config.user_config:load:144 No global config
2023-06-30 05:42:34,987 DEBUG 1274:152c ggshield.core.config.user_config:load:153 No local config
2023-06-30 05:42:34,988 DEBUG 1274:152c ggshield.core.git_shell:git:80 command=['rev-parse', '--show-toplevel']
2023-06-30 05:42:34,990 DEBUG 1274:152c ggshield.core.git_shell:_get_git_path:44 Found git at C:\Program Files\Git\mingw64\bin\git.EXE
2023-06-30 05:42:35,007 DEBUG 1274:152c ggshield.core.git_shell:git:80 command=['rev-parse', '--show-toplevel']
2023-06-30 05:42:35,020 DEBUG 1274:152c ggshield.core.config.config:api_key:149 Using API key from $GITGUARDIAN_API_KEY
2023-06-30 05:42:35,034 DEBUG 1274:152c urllib3.connectionpool:_new_conn:1048 Starting new HTTPS connection (1): api.gitguardian.com:443
 2023-06-30 05:42:35,592 DEBUG 1274:152c urllib3.connectionpool:_make_request:546 https://api.gitguardian.com:443 "GET /v1/health HTTP/1.1" 200 27
2023-06-30 05:42:35,596 DEBUG 1274:152c pygitguardian.client:request:204 method=get endpoint=health status_code=200 duration=0.573461
API URL: https://api.gitguardian.com
Status: healthy
App version: v2.33.0
Secrets engine version: 2.92.0
 
2023-06-30 05:42:35,598 DEBUG 1274:152c ggshield.core.check_updates:check_for_updates:60 Checking the latest released version of ggshield...
2023-06-30 05:42:35,602 DEBUG 1274:152c urllib3.connectionpool:_new_conn:1048 Starting new HTTPS connection (1): api.github.com:443
2023-06-30 05:42:35,896 DEBUG 1274:152c urllib3.connectionpool:_make_request:546 https://api.github.com:443 "GET /repos/GitGuardian/GGShield/releases/latest HTTP/1.1" 200 975
ggshield secret scan docker artifactory.our.domain/docker/dotnet-framework:1.0 --docker-timeout=1200 --allow-self-signed
Saving docker image... OK
...
Skipped \Hives\System_Delta: can't detect encoding
Scanning... ---------------          67% 1767 files scanned out of 2630 0:00:06
 
 
Error: Scanning failed: HTTPSConnectionPool(host='api.gitguardian.com', port=443): Max retries exceeded with url: /v1/multiscan?ignore_known_secrets=True (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))
WARNING: GitGuardian Shield returned non-zero exit code

testworksau avatar Jun 30 '23 07:06 testworksau

I think there is an issue with the way the ggshield branch has been installed: as long as --debug is set, the scan_threads=N log message should appear whether or not GG_MAX_WORKERS has been defined.

Based on the log output you posted, the path to the ggshield command is Z:\docker-image-builder\ggshield\Scripts\ggshield. Is it possible that the pip install command installed ggshield somewhere else?

agateau-gg avatar Jul 03 '23 14:07 agateau-gg

Ah, that is my bad @agateau-gg - I didn't have the --debug command on the actual scan command.

Saving docker image... OK
2023-07-04 07:51:46,770 DEBUG 2538:195c urllib3.connectionpool:_new_conn:1048 Starting new HTTPS connection (1): api.gitguardian.com:443
2023-07-04 07:51:47,226 DEBUG 2538:195c urllib3.connectionpool:_make_request:546 https://api.gitguardian.com:443 "GET /v1/metadata HTTP/1.1" 200 401
2023-07-04 07:51:47,236 DEBUG 2538:195c pygitguardian.client:request:204 method=get endpoint=metadata status_code=200 duration=0.469432
Scanning Docker config
2023-07-04 07:51:47,263 DEBUG 2538:195c ggshield.secret.secret_scanner:scan:102 files=<ggshield.secret.secret_scanner.SecretScanner object at 0x00000266DF5FC6D0> command_id=76193842-473c-42f5-a59a-ad1bb3674f4c scan_threads=1
2023-07-04 07:51:47,636 DEBUG 2538:199c urllib3.connectionpool:_make_request:546 https://api.gitguardian.com:443 "POST /v1/multiscan?ignore_known_secrets=True HTTP/1.1" 200 108
2023-07-04 07:51:47,638 DEBUG 2538:199c pygitguardian.client:request:204 method=post endpoint=multiscan status_code=200 duration=0.374216
Scanning... ----------------------------- 100% 1 files scanned out of 1 0:00:00
Scanning layer sha256:ab13d27ccff3aa9ff945291a48a55a1bfc2c121b5f6437216a4e74501538a1c7
2023-07-04 07:51:49,926 DEBUG 2538:195c ggshield.secret.secret_scanner:scan:102 files=<ggshield.secret.secret_scanner.SecretScanner object at 0x00000266DF5FC6D0> command_id=76193842-473c-42f5-a59a-ad1bb3674f4c scan_threads=1
Skipped
\Files\ProgramData\USOShared\Logs\System\UpdateSessionOrchestration_Temp.1.etl:
can't detect encoding
...
23-07-04 07:52:22,257 DEBUG 2538:1bb0 pygitguardian.client:request:204 method=post endpoint=multiscan status_code=200 duration=0.336224
2023-07-04 07:52:22,536 DEBUG 2538:1bb0 urllib3.connectionpool:_make_request:546 https://api.gitguardian.com:443 "POST /v1/multiscan?ignore_known_secrets=True HTTP/1.1" 200 None
2023-07-04 07:52:22,544 DEBUG 2538:1bb0 pygitguardian.client:request:204 method=post endpoint=multiscan status_code=200 duration=0.284853
2023-07-04 07:52:22,725 DEBUG 2538:1bb0 urllib3.connectionpool:_make_request:546 https://api.gitguardian.com:443 "POST /v1/multiscan?ignore_known_secrets=True HTTP/1.1" 200 322
2023-07-04 07:52:22,732 DEBUG 2538:1bb0 pygitguardian.client:request:204 method=post endpoint=multiscan status_code=200 duration=0.187401
Scanning... -----------------        78% 2047 files scanned out of 2630 0:00:04
 
 
Error: Scanning failed: HTTPSConnectionPool(host='api.gitguardian.com', port=443): Max retries exceeded with url: /v1/multiscan?ignore_known_secrets=True (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))
2023-07-04 07:52:23,498 DEBUG 2538:195c ggshield.cmd.main:exit_code:50 scan exit_code=128

testworksau avatar Jul 04 '23 08:07 testworksau

OK, I pushed another branch: agateau/debug-555 with more changes:

  • You can now set the GG_MAX_DOCS environment variable to define the number of documents scanned at once (ggshield sends the document by batches).
  • The scans were not fully synchronous before: multiple HTTP requests could be sent at once. This is no longer the case in this branch.
  • ggshield logs the name of all scanned files and their size (careful: you may want to strip this out when pasting logs here, or send them to me via email).

I'd like to investigate if the problem is:

  • A. caused by multiple HTTP requests, or
  • B. caused by one specific file in your Docker image (I don't see why, but that's what investigation is for...).

Can you start with running the scan from the agateau/debug-555 branch with GG_MAX_WORKERS to 1?

If it passes then A is probably the issue.

If it still does not pass, can you run the scan with GG_MAX_WORKERS and GG_MAX_DOCS both set to 1, at least 2 or 3 times?

If the scan always fail on the same file, then the problem is B and we need to investigate what's wrong with that particular file.

Warning: scanning with GG_MAX_WORKERS=1 and GG_MAX_DOCS=1 is going to be slow...

agateau-gg avatar Jul 04 '23 16:07 agateau-gg

Hi @agateau-gg 👋🏼

The scan failed with the SSLEOFError with GG_MAX_WORKERS = 1

The scan failed with different errors with GG_MAX_WORKERS = 1 and GG_MAX_DOCS = 1.

Here's the last snippets of the bottom of each log (I tried 3 times) for the latter scenario; you'll notice effectively the same output, aside from the % completion:

2023-07-06 12:29:14 AEST | Skipped \Files\packages\python.3.8.10\tools\libs\winsound.lib: can't detect
2023-07-06 12:29:14 AEST | encoding
2023-07-06 12:29:14 AEST | Skipped \Hives\DefaultUser_Delta: can't detect encoding
2023-07-06 12:29:15 AEST | Skipped \Hives\Sam_Delta: can't detect encoding
2023-07-06 12:29:15 AEST | Skipped \Hives\Security_Delta: can't detect encoding
2023-07-06 12:29:15 AEST | Skipped \Hives\Software_Delta: can't detect encoding
2023-07-06 12:29:15 AEST | Skipped \Hives\System_Delta: can't detect encoding
2023-07-06 12:29:16 AEST | 2023-07-06 02:29:16,044 ERROR 156c:2910 ggshield.secret.secret_scanner:handle_scan_chunk_error:246 status_code=None detail=file exceeds the maximum allowed size of 10485800B
2023-07-06 12:29:16 AEST | Scanning... ------------            56% 7722 files scanned out of 13695 0:00:01
2023-07-06 12:29:16 AEST |  
2023-07-06 12:29:16 AEST |  
2023-07-06 12:29:16 AEST |  
2023-07-06 12:29:16 AEST |  
2023-07-06 12:29:16 AEST |  
2023-07-06 12:29:16 AEST |  
2023-07-06 12:29:16 AEST | Error: Scanning failed: file exceeds the maximum allowed size of 10485800B
2023-07-06 12:29:17 AEST | 2023-07-06 02:29:17,169 DEBUG 156c:2910 ggshield.cmd.main:exit_code:50 scan exit_code=128

2023-07-06 16:45:22 AEST | Skipped \Files\packages\python.3.8.10\tools\libs\winsound.lib: can't detect
2023-07-06 16:45:22 AEST | encoding
2023-07-06 16:45:23 AEST | Skipped \Hives\DefaultUser_Delta: can't detect encoding
2023-07-06 16:45:23 AEST | Skipped \Hives\Sam_Delta: can't detect encoding
2023-07-06 16:45:23 AEST | Skipped \Hives\Security_Delta: can't detect encoding
2023-07-06 16:45:23 AEST | Skipped \Hives\Software_Delta: can't detect encoding
2023-07-06 16:45:23 AEST | Skipped \Hives\System_Delta: can't detect encoding
2023-07-06 16:45:24 AEST | 2023-07-06 06:45:24,020 ERROR 1a28:1454 ggshield.secret.secret_scanner:handle_scan_chunk_error:246 status_code=None detail=file exceeds the maximum allowed size of 10485800B
2023-07-06 16:45:24 AEST | Scanning... ---------               45% 6116 files scanned out of 13690 -:--:--
2023-07-06 16:45:24 AEST |  
2023-07-06 16:45:24 AEST |  
2023-07-06 16:45:24 AEST |  
2023-07-06 16:45:24 AEST |  
2023-07-06 16:45:24 AEST |  
2023-07-06 16:45:24 AEST |  
2023-07-06 16:45:24 AEST | Error: Scanning failed: file exceeds the maximum allowed size of 10485800B
2023-07-06 16:45:24 AEST | 2023-07-06 06:45:24,809 DEBUG 1a28:1454 ggshield.cmd.main:exit_code:50 scan exit_code=128

2023-07-06 18:00:13 AEST | Skipped \Files\packages\python.3.8.10\tools\libs\winsound.lib: can't detect
2023-07-06 18:00:13 AEST | encoding
2023-07-06 18:00:13 AEST | Skipped \Hives\DefaultUser_Delta: can't detect encoding
2023-07-06 18:00:13 AEST | Skipped \Hives\Sam_Delta: can't detect encoding
2023-07-06 18:00:13 AEST | Skipped \Hives\Security_Delta: can't detect encoding
2023-07-06 18:00:14 AEST | Skipped \Hives\Software_Delta: can't detect encoding
2023-07-06 18:00:14 AEST | Skipped \Hives\System_Delta: can't detect encoding
2023-07-06 18:00:14 AEST | 2023-07-06 08:00:14,675 ERROR 1b5c:19f8 ggshield.secret.secret_scanner:handle_scan_chunk_error:246 status_code=None detail=file exceeds the maximum allowed size of 10485800B
2023-07-06 18:00:14 AEST | Scanning... --                      13% 1732 files scanned out of 13690 0:00:36
2023-07-06 18:00:14 AEST |  
2023-07-06 18:00:14 AEST |  
2023-07-06 18:00:14 AEST |  
2023-07-06 18:00:14 AEST |  
2023-07-06 18:00:14 AEST |  
2023-07-06 18:00:14 AEST |  
2023-07-06 18:00:14 AEST | Error: Scanning failed: file exceeds the maximum allowed size of 10485800B
2023-07-06 18:00:15 AEST | 2023-07-06 08:00:15,458 DEBUG 1b5c:19f8 ggshield.cmd.main:exit_code:50 scan exit_code=128

On a side note, it would be really great if we could skip the first layer or so of known images. I saw some changes recently re. caching scan results, but that likely won't help us with our ephemeral build agents (unless you guys cache the results for each layer hash on your side).

testworksau avatar Jul 06 '23 23:07 testworksau

That's interesting: the "maximum allowed size" bug is #561, which we plan to work on soon. I wonder if the SSL error could be caused by that 🤔. I am going to make some changes to the debug branch to artificially restrict this size so that you don't hit that bug anymore.

On a side note, it would be really great if we could skip the first layer or so of known images. I saw some changes recently re. caching scan results, but that likely won't help us with our ephemeral build agents (unless you guys cache the results for each layer hash on your side).

Skipping the first layer was our initial plan, but it felt less efficient if other layers add a lot of tools to the image as these layers would still be scanned every time. CI systems often have the ability to make a given directory persist between runs, even with ephemeral build agents. Have you looked into this?

agateau-gg avatar Jul 07 '23 08:07 agateau-gg

Just pushed a new commit to the agateau/debug-555 branch which should avoid the bug. I just noticed you are scanning large documents (more than 10 MB). I wonder if that could be related.

agateau-gg avatar Jul 07 '23 10:07 agateau-gg

Sorry @agateau-gg , I still get the same error. Additionally, I can't find the "to avoid bug" output in the build log

t=1688969928593  Resolved https://github.com/gitguardian/ggshield to commit 4e9c8668748c8b2962408ba4a427f632b4a0f29c
...
t=1688975316337Skipped \Hives\DefaultUser_Delta: can't detect encoding
t=1688975316342Skipped \Hives\Sam_Delta: can't detect encoding
t=1688975316348Skipped \Hives\Security_Delta: can't detect encoding
t=1688975317215Skipped \Hives\Software_Delta: can't detect encoding
t=1688975317262Skipped \Hives\System_Delta: can't detect encoding
t=16889753173332023-07-10 07:48:37,326 ERROR 1a94:ef0 ggshield.secret.secret_scanner:handle_scan_chunk_error:256 status_code=None detail=file exceeds the maximum allowed size of 10485800B
t=1688975317347Scanning... ------------            57% 7809 files scanned out of 13690 -:--:--
t=1688975317347
t=1688975317347
t=1688975317347
t=1688975317347
t=1688975317347
t=1688975317347
t=1688975317347Error: Scanning failed: file exceeds the maximum allowed size of 10485800B
t=16889753181092023-07-10 07:48:38,095 DEBUG 1a94:ef0 ggshield.cmd.main:exit_code:50 scan exit_code=128

testworksau avatar Jul 10 '23 22:07 testworksau

I found another issue which would cause the "Scanning failed: file exceeds the maximum allowed size of 10485800B". I pushed another commit to agateau/debug-555 to workaround it. Can you give it a try?

agateau-gg avatar Jul 11 '23 11:07 agateau-gg

Same again, sorry @agateau-gg

2023-07-13 17:38:40 AEST | Skipped \Hives\DefaultUser_Delta: can't detect encoding
2023-07-13 17:38:40 AEST | Skipped \Hives\Sam_Delta: can't detect encoding
2023-07-13 17:38:40 AEST | Skipped \Hives\Security_Delta: can't detect encoding
2023-07-13 17:38:41 AEST | Skipped \Hives\Software_Delta: can't detect encoding
2023-07-13 17:38:41 AEST | Skipped \Hives\System_Delta: can't detect encoding
2023-07-13 17:38:41 AEST | 2023-07-13 07:38:41,848 ERROR 2f8c:1df4 ggshield.secret.secret_scanner:handle_scan_chunk_error:263 status_code=None detail=HTTPSConnectionPool(host='api.gitguardian.com', port=443): Max retries exceeded with url: /v1/multiscan?ignore_known_secrets=True (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))
2023-07-13 17:38:41 AEST | Scanning... -----                   26% 3571 files scanned out of 13692 0:00:01
2023-07-13 17:38:41 AEST |  
2023-07-13 17:38:41 AEST |  
2023-07-13 17:38:41 AEST |  
2023-07-13 17:38:41 AEST |  
2023-07-13 17:38:41 AEST |  
2023-07-13 17:38:41 AEST |  
2023-07-13 17:38:41 AEST | Error: Scanning failed: HTTPSConnectionPool(host='api.gitguardian.com', port=443): Max retries exceeded with url: /v1/multiscan?ignore_known_secrets=True (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))
2023-07-13 17:38:42 AEST | 2023-07-13 07:38:42,226 DEBUG 2f8c:1df4 ggshield.cmd.main:exit_code:50 scan exit_code=128

testworksau avatar Jul 13 '23 10:07 testworksau

I'd like to eliminate the possibility of an SSL connection issue between your server and GitGuardian.

I attached a small Python script call netcheck.py to this comment. It works by repeatedly sending a GET request to our API server using a variety of methods.

Can you install it in the same virtualenv as ggshield, then run it with python netcheck.py -r 20 (or a higher number) and report the result?

netcheck.zip

agateau-gg avatar Aug 03 '23 16:08 agateau-gg

Here's what that returned @agateau-gg

03:34:59 [INFO] Starting test using https://api.gitguardian.com
03:34:59 [INFO] Run 1/20
03:34:59 [INFO] Testing with requests (verify=True)
03:34:59 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
03:35:00 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
03:35:00 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
03:35:00 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
03:35:00 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
03:35:01 [INFO] OK
03:35:01 [INFO] Testing with requests (verify=False)
03:35:01 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
C:\Python311\Lib\site-packages\urllib3\connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.gitguardian.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
03:35:01 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
03:35:01 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
03:35:02 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
C:\Python311\Lib\site-packages\urllib3\connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.gitguardian.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
03:35:02 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
03:35:02 [INFO] OK
03:35:02 [INFO] Defining REQUESTS_CA_BUNDLE to C:\<directory>\builds\build-windows-i-<id>-2\docker-image-builder\scripts\cacert.pem
03:35:02 [INFO] Testing with requests (verify=True)
03:35:02 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
03:35:03 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
03:35:03 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
03:35:03 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
03:35:03 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
03:35:04 [INFO] OK
03:35:04 [INFO] Testing with urllib
03:35:04 [ERROR] Network error
Traceback (most recent call last):
  File "C:\Python311\Lib\urllib\request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "C:\Python311\Lib\http\client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Python311\Lib\http\client.py", line 1332, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Python311\Lib\http\client.py", line 1281, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Python311\Lib\http\client.py", line 1041, in _send_output
    self.send(msg)
  File "C:\Python311\Lib\http\client.py", line 979, in send
    self.connect()
  File "C:\Python311\Lib\http\client.py", line 1458, in connect
    self.sock = self._context.wrap_socket(self.sock,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\ssl.py", line 1075, in _create
    self.do_handshake()
  File "C:\Python311\Lib\ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\<directory>\builds\build-windows-i-<id>-2\docker-image-builder\scripts\netcheck.py", line 40, in run_test
    response = request.urlopen(url)
               ^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\urllib\request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\urllib\request.py", line 519, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\urllib\request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\urllib\request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "C:\Python311\Lib\urllib\request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\urllib\request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)>
03:35:04 [INFO] Run 2/20

testworksau avatar Aug 04 '23 03:08 testworksau

Thanks, the urllib error can be ignored because ggshield only uses requests.

I created a git repository for netcheck and made some improvements to it. Can you get it from https://github.com/agateau-gg/netcheck and re-run the test with -m requests -r 20?

agateau-gg avatar Aug 04 '23 13:08 agateau-gg

Here you go @agateau-gg

Running netcheck
Collecting requests
  Downloading requests-2.31.0-py3-none-any.whl (62 kB)
     ---------------------------------------- 62.6/62.6 kB 3.3 MB/s eta 0:00:00
Collecting charset-normalizer<4,>=2 (from requests)
  Downloading charset_normalizer-3.2.0-cp311-cp311-win_amd64.whl (96 kB)
     ---------------------------------------- 96.6/96.6 kB ? eta 0:00:00
Collecting idna<4,>=2.5 (from requests)
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     ---------------------------------------- 61.5/61.5 kB ? eta 0:00:00
Collecting urllib3<3,>=1.21.1 (from requests)
  Downloading urllib3-2.0.4-py3-none-any.whl (123 kB)
     ---------------------------------------- 123.9/123.9 kB ? eta 0:00:00
Collecting certifi>=2017.4.17 (from requests)
  Downloading certifi-2023.7.22-py3-none-any.whl (158 kB)
     ---------------------------------------- 158.3/158.3 kB ? eta 0:00:00
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests
Successfully installed certifi-2023.7.22 charset-normalizer-3.2.0 idna-3.4 requests-2.31.0 urllib3-2.0.4
[notice] A new release of pip is available: 23.1.2 -> 23.2.1
[notice] To update, run: python.exe -m pip install --upgrade pip
04:51:13 [INFO] Starting test using https://api.gitguardian.com
04:51:13 [INFO] Run 1/20
04:51:13 [INFO] Testing mode=requests
04:51:13 [INFO] - Testing SSL
04:51:13 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:14 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:14 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:14 [INFO] - Testing url=https://api.gitguardian.com
04:51:14 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:14 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:14 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:14 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:15 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:15 [INFO] Request took 1.6s
04:51:15 [INFO] OK
04:51:15 [INFO] Run 2/20
04:51:15 [INFO] Testing mode=requests
04:51:15 [INFO] - Testing SSL
04:51:15 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:16 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:16 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:16 [INFO] - Testing url=https://api.gitguardian.com
04:51:16 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:16 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:16 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:17 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:17 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:17 [INFO] Request took 1.4s
04:51:17 [INFO] OK
04:51:17 [INFO] Run 3/20
04:51:17 [INFO] Testing mode=requests
04:51:17 [INFO] - Testing SSL
04:51:17 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:18 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:18 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:18 [INFO] - Testing url=https://api.gitguardian.com
04:51:18 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:19 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:19 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:19 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:19 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:20 [INFO] Request took 1.4s
04:51:20 [INFO] OK
04:51:20 [INFO] Run 4/20
04:51:20 [INFO] Testing mode=requests
04:51:20 [INFO] - Testing SSL
04:51:20 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:20 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:20 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:20 [INFO] - Testing url=https://api.gitguardian.com
04:51:20 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:21 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:21 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:21 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:21 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:22 [INFO] Request took 1.4s
04:51:22 [INFO] OK
04:51:22 [INFO] Run 5/20
04:51:22 [INFO] Testing mode=requests
04:51:22 [INFO] - Testing SSL
04:51:22 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:23 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:23 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:23 [INFO] - Testing url=https://api.gitguardian.com
04:51:23 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:23 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:23 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:23 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:24 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:24 [INFO] Request took 1.4s
04:51:24 [INFO] OK
04:51:24 [INFO] Run 6/20
04:51:24 [INFO] Testing mode=requests
04:51:24 [INFO] - Testing SSL
04:51:24 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:25 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:25 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:25 [INFO] - Testing url=https://api.gitguardian.com
04:51:25 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:25 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:25 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:25 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:26 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:26 [INFO] Request took 1.4s
04:51:26 [INFO] OK
04:51:26 [INFO] Run 7/20
04:51:26 [INFO] Testing mode=requests
04:51:26 [INFO] - Testing SSL
04:51:26 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:27 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:27 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:27 [INFO] - Testing url=https://api.gitguardian.com
04:51:27 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:27 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:27 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:28 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:28 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:28 [INFO] Request took 1.4s
04:51:28 [INFO] OK
04:51:28 [INFO] Run 8/20
04:51:28 [INFO] Testing mode=requests
04:51:28 [INFO] - Testing SSL
04:51:28 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:29 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:29 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:29 [INFO] - Testing url=https://api.gitguardian.com
04:51:29 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:30 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:30 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:30 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:30 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:31 [INFO] Request took 1.4s
04:51:31 [INFO] OK
04:51:31 [INFO] Run 9/20
04:51:31 [INFO] Testing mode=requests
04:51:31 [INFO] - Testing SSL
04:51:31 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:31 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:31 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:31 [INFO] - Testing url=https://api.gitguardian.com
04:51:31 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:32 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:32 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:32 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:32 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:33 [INFO] Request took 1.4s
04:51:33 [INFO] OK
04:51:33 [INFO] Run 10/20
04:51:33 [INFO] Testing mode=requests
04:51:33 [INFO] - Testing SSL
04:51:33 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:34 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:34 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:34 [INFO] - Testing url=https://api.gitguardian.com
04:51:34 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:34 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:34 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:34 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:35 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:35 [INFO] Request took 1.4s
04:51:35 [INFO] OK
04:51:35 [INFO] Run 11/20
04:51:35 [INFO] Testing mode=requests
04:51:35 [INFO] - Testing SSL
04:51:35 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:36 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:36 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:36 [INFO] - Testing url=https://api.gitguardian.com
04:51:36 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:36 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:36 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:37 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:37 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:37 [INFO] Request took 1.5s
04:51:37 [INFO] OK
04:51:37 [INFO] Run 12/20
04:51:37 [INFO] Testing mode=requests
04:51:37 [INFO] - Testing SSL
04:51:37 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:38 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:38 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:38 [INFO] - Testing url=https://api.gitguardian.com
04:51:38 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:38 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:38 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:39 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:39 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:39 [INFO] Request took 1.4s
04:51:39 [INFO] OK
04:51:39 [INFO] Run 13/20
04:51:39 [INFO] Testing mode=requests
04:51:39 [INFO] - Testing SSL
04:51:39 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:40 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:40 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:40 [INFO] - Testing url=https://api.gitguardian.com
04:51:40 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:41 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:41 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:41 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:41 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:42 [INFO] Request took 1.4s
04:51:42 [INFO] OK
04:51:42 [INFO] Run 14/20
04:51:42 [INFO] Testing mode=requests
04:51:42 [INFO] - Testing SSL
04:51:42 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:42 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:42 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:42 [INFO] - Testing url=https://api.gitguardian.com
04:51:42 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:43 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:43 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:43 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:43 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:44 [INFO] Request took 1.4s
04:51:44 [INFO] OK
04:51:44 [INFO] Run 15/20
04:51:44 [INFO] Testing mode=requests
04:51:44 [INFO] - Testing SSL
04:51:44 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:45 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:45 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:45 [INFO] - Testing url=https://api.gitguardian.com
04:51:45 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:45 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:45 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:45 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:46 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:46 [INFO] Request took 1.4s
04:51:46 [INFO] OK
04:51:46 [INFO] Run 16/20
04:51:46 [INFO] Testing mode=requests
04:51:46 [INFO] - Testing SSL
04:51:46 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:47 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:47 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:47 [INFO] - Testing url=https://api.gitguardian.com
04:51:47 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:47 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:47 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:48 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:48 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:48 [INFO] Request took 1.4s
04:51:48 [INFO] OK
04:51:48 [INFO] Run 17/20
04:51:48 [INFO] Testing mode=requests
04:51:48 [INFO] - Testing SSL
04:51:48 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:49 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:49 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:49 [INFO] - Testing url=https://api.gitguardian.com
04:51:49 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:49 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:49 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:50 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:50 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:50 [INFO] Request took 1.4s
04:51:50 [INFO] OK
04:51:50 [INFO] Run 18/20
04:51:50 [INFO] Testing mode=requests
04:51:50 [INFO] - Testing SSL
04:51:50 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:51 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:51 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:51 [INFO] - Testing url=https://api.gitguardian.com
04:51:51 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:52 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:52 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:52 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:52 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:53 [INFO] Request took 1.4s
04:51:53 [INFO] OK
04:51:53 [INFO] Run 19/20
04:51:53 [INFO] Testing mode=requests
04:51:53 [INFO] - Testing SSL
04:51:53 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:53 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:53 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:53 [INFO] - Testing url=https://api.gitguardian.com
04:51:53 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:54 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:54 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:54 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:54 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:55 [INFO] Request took 1.4s
04:51:55 [INFO] OK
04:51:55 [INFO] Run 20/20
04:51:55 [INFO] Testing mode=requests
04:51:55 [INFO] - Testing SSL
04:51:55 [DEBUG] Starting new HTTPS connection (1): www.howsmyssl.com:443
04:51:56 [DEBUG] https://www.howsmyssl.com:443 "GET /a/check HTTP/1.1" 200 352
04:51:56 [INFO] {
  "given_cipher_suites": [
    "TLS_AES_256_GCM_SHA384",
    "TLS_CHACHA20_POLY1305_SHA256",
    "TLS_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256",
    "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
    "TLS_EMPTY_RENEGOTIATION_INFO_SCSV"
  ],
  "ephemeral_keys_supported": true,
  "session_ticket_supported": false,
  "tls_compression_supported": false,
  "unknown_cipher_suite_supported": false,
  "beast_vuln": false,
  "able_to_detect_n_minus_one_splitting": false,
  "insecure_cipher_suites": {},
  "tls_version": "TLS 1.3",
  "rating": "Probably Okay"
}
04:51:56 [INFO] - Testing url=https://api.gitguardian.com
04:51:56 [DEBUG] Starting new HTTPS connection (1): api.gitguardian.com:443
04:51:56 [DEBUG] https://api.gitguardian.com:443 "GET / HTTP/1.1" 301 162
04:51:56 [DEBUG] Starting new HTTP connection (1): api.gitguardian.com:80
04:51:56 [DEBUG] http://api.gitguardian.com:80 "GET /docs HTTP/1.1" 301 0
04:51:57 [DEBUG] https://api.gitguardian.com:443 "GET /docs HTTP/1.1" 200 None
04:51:57 [INFO] Request took 1.4s
04:51:57 [INFO] OK
04:51:57 [INFO] Errors: 0

testworksau avatar Aug 07 '23 04:08 testworksau

OK thanks, so netcheck found nothing 😞. At least this culprit has been eliminated.

Another possible culprit might be the document size. From what I see in your log output, your server is configured with a maximum doc size of 10,485,800. That's quite large.

I just rebased the agateau/debug-555 branch on top of main and added yet another environment variable: GG_MAX_DOC_SIZE. This variable makes it possible to set a smaller size limit without reconfiguring your server.

Can you try running a scan with GG_MAX_DOC_SIZE set to 1048576 (the default value) and see if it works better? If it does then running with an increasing max doc size can help us identifying the tipping value.

I am going to do a similar test on my side.

agateau-gg avatar Aug 07 '23 16:08 agateau-gg

Is there a new dep, or has one of the python deps removed their dependency on cryptography?

  File "C:\builds\docker-image-builder\ggshield\Lib\site-packages\ggshield\hmsl\client.py", line 12, in <module>
    from .crypto import decrypt, make_hint
  File "C:\builds\docker-image-builder\ggshield\Lib\site-packages\ggshield\hmsl\crypto.py", line 6, in <module>
    from cryptography.exceptions import InvalidTag
ModuleNotFoundError: No module named 'cryptography'

testworksau avatar Aug 08 '23 07:08 testworksau

This is odd: the dependency on cryptography has been added as optional in 1.17.2. It is still optional as of now. In any case, the dependency is going to be mandatory in 1.18.0, so you might as well install it.

agateau-gg avatar Aug 08 '23 09:08 agateau-gg

🎉 🎉 🎉 🎉 🎉

2023-08-09 07:51:26,542 DEBUG 2290:2950 urllib3.connectionpool:_make_request:546 https://api.gitguardian.com:443 "POST /v1/multiscan?ignore_known_secrets=True HTTP/1.1" 200 None
2023-08-09 07:51:26,558 DEBUG 2290:2950 pygitguardian.client:request:209 method=post endpoint=multiscan status_code=200 duration=0.249610
Scanning... --------------------- 100% 13691 files scanned out of 13691 0:00:00
No secrets have been found
2023-08-09 07:51:27,396 DEBUG 2290:2354 ggshield.cmd.main:exit_code:50 scan exit_code=0

FWIW I think our max doc size was set by the support team as the scanner was causing windows docker image builds to fail with a non-zero exit code.

testworksau avatar Aug 09 '23 08:08 testworksau

Interesting! Can you try a few different values for the maximum size so that we have a rough idea of the point it starts to cause problems?

agateau-gg avatar Aug 09 '23 16:08 agateau-gg

Here you go @agateau-gg

GG_MAX_DOC_SIZE=5242880

Scanning... ----------------------   96% 2535 files scanned out of 2635 0:00:01
Error: Scanning failed: HTTPSConnectionPool(host='api.gitguardian.com', port=443): Max retries exceeded with url: /v1/multiscan?ignore_known_secrets=True (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))
2023-08-10 00:48:45,693 DEBUG 764:1ee4 ggshield.cmd.main:exit_code:50 scan exit_code=128

GG_MAX_DOC_SIZE=3145728

Scanning... --------------           63% 1652 files scanned out of 2635 0:00:18
Error: Scanning failed: HTTPSConnectionPool(host='api.gitguardian.com', port=443): Max retries exceeded with url: /v1/multiscan?ignore_known_secrets=True (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))
2023-08-10 02:17:38,997 DEBUG 1e4:1450 ggshield.cmd.main:exit_code:50 scan exit_code=128

GG_MAX_DOC_SIZE=2097152

Scanning... --------------------     91% 2395 files scanned out of 2635 0:00:03
Error: Scanning failed: HTTPSConnectionPool(host='api.gitguardian.com', port=443): Max retries exceeded with url: /v1/multiscan?ignore_known_secrets=True (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))
2023-08-10 04:33:47,049 DEBUG 10d4:1084 ggshield.cmd.main:exit_code:50 scan exit_code=128

GG_MAX_DOC_SIZE=1572864

2023-08-10 05:21:46,816 DEBUG 195c:161c urllib3.connectionpool:_make_request:546 https://api.gitguardian.com:443 "POST /v1/multiscan?ignore_known_secrets=True HTTP/1.1" 200 None
2023-08-10 05:21:46,818 DEBUG 195c:161c pygitguardian.client:request:209 method=post endpoint=multiscan status_code=200 duration=0.220404
Scanning... --------------------- 100% 13694 files scanned out of 13694 0:00:00
No secrets have been found
2023-08-10 05:21:47,170 DEBUG 195c:bd4 ggshield.cmd.main:exit_code:50 scan exit_code=0

testworksau avatar Aug 10 '23 06:08 testworksau

Thanks, looks like it break at quite a low size :(. Going to try to reproduce on our side and see if we can fix this server side. I still don't understand why it causes SSL failures though.

agateau-gg avatar Aug 10 '23 16:08 agateau-gg

I can reproduce the bug on our side, we are going to investigate. We'll keep you posted.

agateau-gg avatar Aug 11 '23 15:08 agateau-gg