[RaspberryPi] OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to httpbin.org:443

[EpicBirb]

Describe the bug Unable to send request to httpbin.org:443 w/ TLS certificate provided

To Reproduce Steps to reproduce the behavior:

run bellow on host machine

1. run make ca-certificates in a folder in the path /home/pi/proxy.py/certificates/ because that is where I put it
2. i ran python3 "/home/pi/proxy.py/proxy_start.py" but you need to run python3 -m proxy --enable-web-server --plugins proxy.plugin.ShortLinkPlugin --port 769 --hostname 0.0.0.0 --ca-key-file "/home/pi/proxy.py/certificates/ca-key.pem" --ca-cert-file "/home/pi/proxy.py/certificates/ca-cert.pem" --ca-signing-key-file "/home/pi/proxy.py/certificates/ca-signing-key.pem"
3. run curl -v -x "address":769 --cacert ca-cert.pem https://httpbin.org/get

Expected behavior expected results from https://github.com/abhinavsingh/proxy.py#tls-interception

Version information

• OS: Raspbian GNU/Linux 10 (buster)
• Browser: cURL
• Device: Raspberry Pi 3 Model B
• proxy.py Version: 2.4.1

Screenshots

[abhinavsingh]

@EpicBirb Can you check if certifi is installed. A few others have reported that on certain environments, certifi might be missing. Interception depends upon a CA bundle, provided by certifi package. Let me know. Thank you

Ref https://github.com/abhinavsingh/proxy.py/issues/1096

[EpicBirb]

[abhinavsingh]

Cool, so that's not an issue in your case. You still see the FileNotFoundError, which simply means intermediate certificates were not generated. We know certifi exists, as expected this should have brought the CA bundle required for interception. Other possible causes can be absence of openssl.

Are you able to generate certificates on your RPi using make https-certificates and similar commands?

[EpicBirb]

pi@birblos:~/proxy.py/certificates $ make https-certificates
# Generate server key
python3 -m proxy.common.pki gen_private_key \
        --private-key-path https-key.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
python3 -m proxy.common.pki remove_passphrase \
        --private-key-path https-key.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
# Generate server certificate
python3 -m proxy.common.pki gen_public_key \
        --private-key-path https-key.pem \
        --public-key-path https-cert.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
pi@birblos:~/proxy.py/certificates $ ls
acert  ca-cert.pem  ca-key.pem  ca-signing-key.pem  https-cert.pem  https-key.pem  Makefile

pi@birblos:~/proxy.py/certificates $ make sign-https-certificates
# Generate CSR request
python3 -m proxy.common.pki gen_csr \
        --csr-path https-csr.pem \
        --private-key-path https-key.pem \
        --public-key-path https-cert.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
# Sign CSR with CA
python3 -m proxy.common.pki sign_csr \
        --csr-path https-csr.pem \
        --crt-path https-signed-cert.pem \
        --hostname localhost \
        --private-key-path ca-key.pem \
        --public-key-path ca-cert.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
pi@birblos:~/proxy.py/certificates $ ls
acert        ca-key.pem          https-cert.pem  https-key.pem          Makefile
ca-cert.pem  ca-signing-key.pem  https-csr.pem   https-signed-cert.pem
pi@birblos:~/proxy.py/certificates $

[EpicBirb]

Can I get a reply?

[abhinavsingh]

Can I get a reply?

Sorry, but this may require me to setup a RPi for reproducing your scenario. Without a reproducible environment we'll end up playing hit-and-trial game.

I don't see any other reason why this ain't working for you (assuming all things are in place). Since you are already on RPi, you can try to debug it locally. Following lines of code are responsible for generating certificates

https://github.com/abhinavsingh/proxy.py/blob/develop/proxy/http/proxy/server.py#L626-L732

What you can try is, debug these methods and see whats wrong. Example, following lines are the key:

1. gen_public_key -- https://github.com/abhinavsingh/proxy.py/blob/develop/proxy/http/proxy/server.py#L672
2. gen_csr -- https://github.com/abhinavsingh/proxy.py/blob/develop/proxy/http/proxy/server.py#L687
3. sign_csr -- https://github.com/abhinavsingh/proxy.py/blob/develop/proxy/http/proxy/server.py#L701

These are same methods that are invoked when you ran make https-certificate and make sign-https-certificates. Try to print resp from these methods and check where are we running into FileNotFoundError. They seem to work fine via make file but what is going wrong when running within our Python environment.

Please let me know.

[EpicBirb]

I've edited somewhere near line 701, 687, 672 to include print(resp)

Here are the newly generated logs:

pi@birblos:~/proxy.py/certificates $ make ca-certificates
# Generate CA key
python3 -m proxy.common.pki gen_private_key \
        --private-key-path ca-key.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
python3 -m proxy.common.pki remove_passphrase \
        --private-key-path ca-key.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
# Generate CA certificate
python3 -m proxy.common.pki gen_public_key \
        --private-key-path ca-key.pem \
        --public-key-path ca-cert.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
# Generate key that will be used to generate domain certificates on the fly
# Generated certificates are then signed with CA certificate / key generated above
python3 -m proxy.common.pki gen_private_key \
        --private-key-path ca-signing-key.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
python3 -m proxy.common.pki remove_passphrase \
        --private-key-path ca-signing-key.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))

pi@birblos:~/proxy.py/certificates $ make https-certificates
# Generate server key
python3 -m proxy.common.pki gen_private_key \
        --private-key-path https-key.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
python3 -m proxy.common.pki remove_passphrase \
        --private-key-path https-key.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
# Generate server certificate
python3 -m proxy.common.pki gen_public_key \
        --private-key-path https-key.pem \
        --public-key-path https-cert.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))

pi@birblos:~/proxy.py/certificates $ make sign-https-certificates
# Generate CSR request
python3 -m proxy.common.pki gen_csr \
        --csr-path https-csr.pem \
        --private-key-path https-key.pem \
        --public-key-path https-cert.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
# Sign CSR with CA
python3 -m proxy.common.pki sign_csr \
        --csr-path https-csr.pem \
        --crt-path https-signed-cert.pem \
        --hostname localhost \
        --private-key-path ca-key.pem \
        --public-key-path ca-cert.pem
/usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'proxy.common.pki' found in sys.modules after import of package 'proxy.common', but prior to execution of 'proxy.common.pki'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))

I've reran the server again, here are the logs:

Server:

root@birblos:/home/pi/proxy.py# python3 ./proxy_start.py
2022-05-05 17:22:44,253 - pid:14441 [I] plugins.load:85 - Loaded plugin proxy.http.proxy.HttpProxyPlugin
2022-05-05 17:22:44,254 - pid:14441 [I] plugins.load:85 - Loaded plugin proxy.http.server.HttpWebServerPlugin
2022-05-05 17:22:44,254 - pid:14441 [I] plugins.load:85 - Loaded plugin proxy.plugin.ShortLinkPlugin
2022-05-05 17:22:44,256 - pid:14441 [I] tcp.listen:82 - Listening on 0.0.0.0:769
2022-05-05 17:22:44,292 - pid:14441 [I] pool.setup:108 - Started 4 acceptors in threaded mode
2022-05-05 17:23:29,432 - pid:14451 [W] handler.handle_readables:236 - Exception when receiving from client connection#14 with reason FileNotFoundError(2, 'No such file or directory')
2022-05-05 17:23:29,434 - pid:14451 [I] server.access_log:384 - 127.0.0.1:44286 - CONNECT httpbin.org:443 - 0 bytes - 64.34ms

Client:

pi@birblos:~/proxy.py/certificates $ curl -v -x 0.0.0.0:769 --cacert ca-cert.pem https://httpbin.org/get
* Expire in 0 ms for 6 (transfer 0x7418b0)
*   Trying 0.0.0.0...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x7418b0)
* Connected to 0.0.0.0 (127.0.0.1) port 769 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to httpbin.org:443
> CONNECT httpbin.org:443 HTTP/1.1
> Host: httpbin.org:443
> User-Agent: curl/7.64.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: ca-cert.pem
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to httpbin.org:443
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to httpbin.org:443

[abhinavsingh]

I'll recommend to run proxy with debug level logging. Above is not helpful in debugging still, because we want to see print(resp) for a request cycle. We don't see any print statements above for the server logs.

[EpicBirb]

I might have revert the print(resp) in the server.py file. I'm going to go back at revert at and run it again, you should see a response within 2 days as such because I think I'm getting sick from my raspberry pi.

[abhinavsingh]

@EpicBirb A similar issue was raised by a Windows user. For them issue was that openssl binary was not in the PATH. They fixed it like so, see https://github.com/abhinavsingh/proxy.py/issues/1146#issuecomment-1123242564

Just curious if you are also seeing same symptoms. Wanted to mention this here. Best

[EpicBirb]

If openssl wasn't in PATH, then I wouldn't be able to execute the command at all. Reference bellow

I've also put back the print(resp) and but the log-level to d and reran the script

doesn't seem to be working because I already connected my pc to the proxy and it didn't show up in the logs :/

I've also probably released that that was stupid because those functions are ment to generate the certificates

[EpicBirb]

oh yeah, sorry for the late response

[EpicBirb]

cam back a while waiting for a response but I might found something that seems to be the issue of this.

Could that RuntimeWarning be the issue why this isn't working?

[abhinavsingh]

@EpicBirb If you see contents within ca-key.pem, then these are harmless. I have never seen them as a cause of error. Certificates are generated just fine, but worth a double check manually :)