micropython-ota-updater OSError: stream operation not supported

HI again!

Unfortunately, I encountered some issues and I believe it has something to do with decoding the Json from GitHub. Here is the full trace. Again, I will try to debug this on my end. It is quite likely that I set it up wrong. But any help will be appreciated. Thanks.

Repo: https://github.com/FalconBFR/esp32otatestv3/

Full Trace:

Updating to version 3...
Downloading version 3
[{'type': 'file', '_links': {'html': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/3/src/app/__init__.py', 'self': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/__init__.py?ref=refs/tags/3', 'git': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/e69de29bb2d1d6434b8b29ae775ad8c2e48c5391'}, 'path': 'src/app/__init__.py', 'size': 0, 'name': '__init__.py', 'git_url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/e69de29bb2d1d6434b8b29ae775ad8c2e48c5391', 'html_url': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/3/src/app/__init__.py', 'download_url': 'https://raw.githubusercontent.com/FalconBFR/esp32otatestv3/refs/tags/3/src/app/__init__.py', 'url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/__init__.py?ref=refs/tags/3', 'sha': 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391'}, {'type': 'file', '_links': {'html': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/3/src/app/httpclientforotaupdater.py', 'self': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/httpclientforotaupdater.py?ref=refs/tags/3', 'git': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/4b6e2779d0caf7a722b04967c1447392596dc627'}, 'path': 'src/app/httpclientforotaupdater.py', 'size': 4898, 'name': 'httpclientforotaupdater.py', 'git_url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/4b6e2779d0caf7a722b04967c1447392596dc627', 'html_url': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/3/src/app/httpclientforotaupdater.py', 'download_url': 'https://raw.githubusercontent.com/FalconBFR/esp32otatestv3/refs/tags/3/src/app/httpclientforotaupdater.py', 'url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/httpclientforotaupdater.py?ref=refs/tags/3', 'sha': '4b6e2779d0caf7a722b04967c1447392596dc627'}, {'type': 'file', '_links': {'html': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/3/src/app/onstart.py', 'self': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/onstart.py?ref=refs/tags/3', 'git': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/de7d314322b6411436ad1683cf3d0dacb75050d9'}, 'path': 'src/app/onstart.py', 'size': 1125, 'name': 'onstart.py', 'git_url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/de7d314322b6411436ad1683cf3d0dacb75050d9', 'html_url': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/3/src/app/onstart.py', 'download_url': 'https://raw.githubusercontent.com/FalconBFR/esp32otatestv3/refs/tags/3/src/app/onstart.py', 'url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/onstart.py?ref=refs/tags/3', 'sha': 'de7d314322b6411436ad1683cf3d0dacb75050d9'}, {'type': 'file', '_links': {'html': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/3/src/app/ota_updater.py', 'self': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/ota_updater.py?ref=refs/tags/3', 'git': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/89ee90e29a5f0f3e50ae2fdf5262c8b1a1ef7391'}, 'path': 'src/app/ota_updater.py', 'size': 10039, 'name': 'ota_updater.py', 'git_url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/89ee90e29a5f0f3e50ae2fdf5262c8b1a1ef7391', 'html_url': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/3/src/app/ota_updater.py', 'download_url': 'https://raw.githubusercontent.com/FalconBFR/esp32otatestv3/refs/tags/3/src/app/ota_updater.py', 'url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/ota_updater.py?ref=refs/tags/3', 'sha': '89ee90e29a5f0f3e50ae2fdf5262c8b1a1ef7391'}, {'type': 'dir', '_links': {'html': 'https://github.com/FalconBFR/esp32otatestv3/tree/refs/tags/3/src/app/updater', 'self': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/updater?ref=refs/tags/3', 'git': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/trees/2bc342fb4681b766015896582ea47921efe1c239'}, 'path': 'src/app/updater', 'size': 0, 'name': 'updater', 'git_url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/trees/2bc342fb4681b766015896582ea47921efe1c239', 'html_url': 'https://github.com/FalconBFR/esp32otatestv3/tree/refs/tags/3/src/app/updater', 'download_url': None, 'url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/updater?ref=refs/tags/3', 'sha': '2bc342fb4681b766015896582ea47921efe1c239'}]
Traceback (most recent call last):
  File "main.py", line 5, in <module>
  File "app/onstart.py", line 34, in <module>
  File "app/onstart.py", line 23, in connectToWifiAndUpdate
  File "app/updater/ota_updater.py", line 78, in install_update_if_available
  File "app/updater/ota_updater.py", line 129, in _download_new_version
  File "app/updater/ota_updater.py", line 137, in _download_all_files
  File "app/updater/httpclientforotaupdater.py", line 42, in json

Note: I did add print statements to the download all files function as I was debugging another issue (which is closed now). Here is my version of the _download_all_files functionalists (Lines 132 to 160)

    def _download_all_files(self, version, sub_dir=''):
        url = 'https://api.github.com/repos/{}/contents{}{}{}?ref=refs/tags/{}'.format(self.github_repo, self.github_src_dir, self.main_dir, sub_dir, version)
        gc.collect() 
        file_list = self.http_client.get(url)
        print(file_list.json())
        for file in file_list.json():
            #self added to debug
            print(file)
            print(type(file))
            print(file['path'])
            temp_file_path = file['path'].replace(self.main_dir + '/', '')
            print("done 1")
            print(temp_file_path)
            temp_file_path = temp_file_path.replace(self.github_src_dir, '')
            print("done 2")
            print(temp_file_path)
            path = self.modulepath(self.new_version_dir + '/' + temp_file_path)
            #path = self.modulepath(self.new_version_dir + '/' + file['path'].replace(self.main_dir + '/', '').replace(self.github_src_dir, ''))
            if file['type'] == 'file':
                gitPath = file['path']
                print('\tDownloading: ', gitPath, 'to', path)
                self._download_file(version, gitPath, path)
            elif file['type'] == 'dir':
                print('Creating dir', path)
                self.mkdir(path)
                self._download_all_files(version, sub_dir + '/' + file['name'])
            gc.collect()

        file_list.close()

Apr 01 '21 11:04 tongclement

Found these 2 posts online: https://forum.micropython.org/viewtopic.php?t=5018&start=10 https://forum.micropython.org/viewtopic.php?p=34317#p34317

Might be useful. Thanks!

Apr 01 '21 11:04 tongclement

More Trace:

<_SSLSocket 3ffec6e0>
<class 'ussl'>
Checking version... 
	Current version:  0.0
	Latest version:  4
Updating to version 4...
Downloading version 4
<_SSLSocket 3ffec6e0>
<class 'ussl'>
[{'type': 'file', '_links': {'html': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/4/src/app/__init__.py', 'self': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/__init__.py?ref=refs/tags/4', 'git': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/447c62121ffec6a6404c4977775e010f81602ffc'}, 'path': 'src/app/__init__.py', 'size': 8, 'name': '__init__.py', 'git_url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/447c62121ffec6a6404c4977775e010f81602ffc', 'html_url': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/4/src/app/__init__.py', 'download_url': 'https://raw.githubusercontent.com/FalconBFR/esp32otatestv3/refs/tags/4/src/app/__init__.py', 'url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/__init__.py?ref=refs/tags/4', 'sha': '447c62121ffec6a6404c4977775e010f81602ffc'}, {'type': 'file', '_links': {'html': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/4/src/app/httpclientforotaupdater.py', 'self': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/httpclientforotaupdater.py?ref=refs/tags/4', 'git': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/4b6e2779d0caf7a722b04967c1447392596dc627'}, 'path': 'src/app/httpclientforotaupdater.py', 'size': 4898, 'name': 'httpclientforotaupdater.py', 'git_url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/4b6e2779d0caf7a722b04967c1447392596dc627', 'html_url': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/4/src/app/httpclientforotaupdater.py', 'download_url': 'https://raw.githubusercontent.com/FalconBFR/esp32otatestv3/refs/tags/4/src/app/httpclientforotaupdater.py', 'url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/httpclientforotaupdater.py?ref=refs/tags/4', 'sha': '4b6e2779d0caf7a722b04967c1447392596dc627'}, {'type': 'file', '_links': {'html': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/4/src/app/onstart.py', 'self': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/onstart.py?ref=refs/tags/4', 'git': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/de7d314322b6411436ad1683cf3d0dacb75050d9'}, 'path': 'src/app/onstart.py', 'size': 1125, 'name': 'onstart.py', 'git_url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/de7d314322b6411436ad1683cf3d0dacb75050d9', 'html_url': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/4/src/app/onstart.py', 'download_url': 'https://raw.githubusercontent.com/FalconBFR/esp32otatestv3/refs/tags/4/src/app/onstart.py', 'url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/onstart.py?ref=refs/tags/4', 'sha': 'de7d314322b6411436ad1683cf3d0dacb75050d9'}, {'type': 'file', '_links': {'html': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/4/src/app/ota_updater.py', 'self': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/ota_updater.py?ref=refs/tags/4', 'git': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/89ee90e29a5f0f3e50ae2fdf5262c8b1a1ef7391'}, 'path': 'src/app/ota_updater.py', 'size': 10039, 'name': 'ota_updater.py', 'git_url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/blobs/89ee90e29a5f0f3e50ae2fdf5262c8b1a1ef7391', 'html_url': 'https://github.com/FalconBFR/esp32otatestv3/blob/refs/tags/4/src/app/ota_updater.py', 'download_url': 'https://raw.githubusercontent.com/FalconBFR/esp32otatestv3/refs/tags/4/src/app/ota_updater.py', 'url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/ota_updater.py?ref=refs/tags/4', 'sha': '89ee90e29a5f0f3e50ae2fdf5262c8b1a1ef7391'}, {'type': 'dir', '_links': {'html': 'https://github.com/FalconBFR/esp32otatestv3/tree/refs/tags/4/src/app/updater', 'self': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/updater?ref=refs/tags/4', 'git': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/trees/4a72155f06860c4c87c7de1d83d40631eef9a966'}, 'path': 'src/app/updater', 'size': 0, 'name': 'updater', 'git_url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/git/trees/4a72155f06860c4c87c7de1d83d40631eef9a966', 'html_url': 'https://github.com/FalconBFR/esp32otatestv3/tree/refs/tags/4/src/app/updater', 'download_url': None, 'url': 'https://api.github.com/repos/FalconBFR/esp32otatestv3/contents/src/app/updater?ref=refs/tags/4', 'sha': '4a72155f06860c4c87c7de1d83d40631eef9a966'}]
None
<class 'NoneType'>
Traceback (most recent call last):
  File "main.py", line 5, in <module>
  File "app/onstart.py", line 34, in <module>
  File "app/onstart.py", line 23, in connectToWifiAndUpdate
  File "app/updater/ota_updater.py", line 78, in install_update_if_available
  File "app/updater/ota_updater.py", line 129, in _download_new_version
  File "app/updater/ota_updater.py", line 137, in _download_all_files
  File "app/updater/httpclientforotaupdater.py", line 44, in json
OSError: stream operation not supported
MicroPython v1.14 on 2021-02-02; ESP32 module with ESP32
Type "help()" for more information.

Apr 01 '21 12:04 tongclement

This might be too late for you, but I stumbled across this issue before solving it myself: You may simply not call .json() of the Response object more than once. Solution: Get rid of the line print(file_list.json()). If you really want to print, save the result of file_list.json() in a variable. Cheers

Aug 23 '22 18:08 arniebarni

This might be too late for you, but I stumbled across this issue before solving it myself: You may simply not call .json() of the Response object more than once. Solution: Get rid of the line print(file_list.json()). If you really want to print, save the result of file_list.json() in a variable. Cheers

Wow many thanks for sharing! I may reconsider micropython again for future projects once OTA works.

Aug 29 '22 17:08 tongclement