vscode-circuitpython icon indicating copy to clipboard operation
vscode-circuitpython copied to clipboard

Published 20 hours ago • verified publisher icon joedevivo

[BUG] Extension doesn't support connecting through proxies for initial download

Open HerbCSO opened this issue 3 years ago • 0 comments

Describe the bug I have a proxy set up on my machine (privoxy) which all requests pass through. When this configuration is active (via setting http_proxy and https_proxy in .bashrc), the initial download of the adafruit libraries fails.

I also get this error: Command 'CircuitPython: Check for latest bundle' resulted in an error (command 'circuitpython.library.fetch' not found)

This is on MacOS 12.2.1, VSCode 1.65.2 and extension 0.1.15.

To Reproduce Steps to reproduce the behavior:

  1. Set http_proxy and https_proxy to a valid proxy host, e.g. both get set to http://127.0.0.1:8118 if you use privoxy - see http://www.privoxy.org/user-manual/quickstart.html for some further instructions for your platform
  2. Install the extension
  3. Open a CircuitPython project (I'm using an Adafruit PyPortal, but I'm guessing anything else should exhibit the same behavior)
  4. Wait for everything to load, try to run the CircuitPython: Check for latest bundle command, it should fail

Expected behavior Extension can load libraries with a proxy active. ;]

Screenshots N/A

Desktop (please complete the following information):

  • OS: [MacOS]
  • Version [12.2.1]

Additional context

I initially saw this error in the Extension Host console:

[2022-03-12 11:14:26.505] [exthost] [error] TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
    at validateString (internal/validators.js:124:11)
    at Object.join (path.js:1039:7)
    at LibraryManager.<anonymous> (/Users/carsten.dreesbach/.vscode/extensions/joedevivo.vscode-circuitpython-0.1.15/out/librarymanager/libraryManager.js:253:39)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/carsten.dreesbach/.vscode/extensions/joedevivo.vscode-circuitpython-0.1.15/out/librarymanager/libraryManager.js:5:58)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

Looking at libraryManager.js on lines 252/253 I see this code:

            let tag = yield this.getLatestBundleTag();
            let localBundleDir = path.join(this.bundleDir, tag);

That then led me to look at getLatestBundleTag() which does this:

    getLatestBundleTag() {
        return __awaiter(this, void 0, void 0, function* () {
            let r = yield axios.default.get('https://github.com/adafruit/Adafruit_CircuitPython_Bundle/releases/latest', { headers: { 'Accept': 'application/json' } });
            return yield r.data.tag_name;
        });
    }

So I said to myself: let's try checking that with curl:

curl -H "Accept: application/json" 'https://github.com/adafruit/Adafruit_CircuitPython_Bundle/releases/latest'
<html><body>You are being <a href="https://github.com/adafruit/Adafruit_CircuitPython_Bundle/releases/tag/20220312">redirected</a>.</body></html>%

A-HA! Following the redirect with -L, I now get the desired JSON response:

curl -LH "Accept: application/json" 'https://github.com/adafruit/Adafruit_CircuitPython_Bundle/releases/latest'
{"id":61660790,"tag_name":"20220312","update_url":"/adafruit/Adafruit_CircuitPython_Bundle/releases/tag/20220312","update_authenticity_token":"-wHZMwbd0fbysIC9YDWVEVyKMF--FoG8N6J1BMQuM9ObpUwpchs33hJujyx_r4aybmSHHGEKnr62lEkcGMEvYQ","delete_url":"/adafruit/Adafruit_CircuitPython_Bundle/releases/tag/20220312","delete_authenticity_token":"MJnkkuEWEhcl07LEtCBh1ke9-O5CRJ76kXLJUGsLIk-S0v8iO64iYtAVbWCClxDsY152u8GHA471NHo6YM_V1A","edit_url":"/adafruit/Adafruit_CircuitPython_Bundle/releases/edit/20220312"}%

As stated above, this is due to my connections going through privoxy, so they land on the proxy first, then get redirected.

Only problem now is, I have no idea how to really fix this! I kinda worked around the first issue by hacking getLatestBundleTag() to return yield r.data.tag_name || '20220312';, which then got me past the initial error, but when it tried to download that bundle it ended up throwing this error:

ERR FILE_ENDED: Error: FILE_ENDED
    at Parse.pull (/Users/carsten.dreesbach/.vscode/extensions/joedevivo.vscode-circuitpython-0.1.15/node_modules/unzipper/lib/PullStream.js:83:28)
    at Parse.emit (events.js:315:20)
    at Parse.EventEmitter.emit (domain.js:467:12)
    at Parse.<anonymous> (/Users/carsten.dreesbach/.vscode/extensions/joedevivo.vscode-circuitpython-0.1.15/node_modules/unzipper/lib/PullStream.js:20:10)
    at Parse.emit (events.js:327:22)
    at Parse.EventEmitter.emit (domain.js:467:12)
    at finish (internal/streams/writable.js:657:10)
    at processTicksAndRejections (internal/process/task_queues.js:80:21)

...and this is where I'm completely stuck now (I'm not a JavaScript programmer and have no idea how to make all of this work properly with my proxy).

Unsetting http_proxy and https_proxy env variables completely kinda lets it work. I still get nothing back from the initial request in getLatestBundleTag() and have to keep the return yield r.data.tag_name || '20220312'; hack in place so it's able to download the libraries (I haven't figured out why that initial request still fails). After it has managed to do that at least once, it seems to be working OK for now (but I've just started using this, not sure if there are other issues I haven't hit yet).

PS: Thank you for putting this together, it's nice to have this (the Mu editor they recommend is a bit basic for my taste).

HerbCSO avatar Mar 12 '22 18:03 HerbCSO