spksrc icon indicating copy to clipboard operation
spksrc copied to clipboard

Published 20 hours ago • verified publisher icon SynoCommunity

META: Python 3.10 / 3.11 package status

Open th0ma7 opened this issue 3 years ago • 38 comments

Python 3.10 package status

High-level python PR

  • [x] python310 (3.10.1) #5044
  • [ ] pre-download wheels #4971
  • [x] mysqlclient wheel #5068
  • [x] avoid error for package without wheel #5092
  • [x] python310 (3.10.2) #5101
  • [x] #5169
  • [x] python310 (3.10.4) #5171
  • [x] #5175
  • [x] Refactor crossenv spksrc.wheel.mk #5180
  • [x] rust wheel support & update to python 3.10.8 #5435

Packages formally using python 3.x

PACKAGE Python 3.10 migration Published Python 3.11 migration Published 3.11 COMMENT
bazarr ✔️ ✔️ Follow-up update #5139 (@smaarn)
beets ✔️ ✔️ #4946
borgbackup ✔️ ✔️ #5048 (co-author @sunlei)
Fixed: https://github.com/borgbackup/borg/issues/6105
domoticz Version online uses python38
duplicity ✔️ ✔️ #5054
fishnet ❌️ ❌️ @mayweed Rewritten in Rust (removal #5216)
flexget ✔️ ✔️ #5176
homeassistant ✔️ ✔️ #5579, #5096, ARMv5 support pending: https://github.com/numpy/numpy/issues/20664
Looks like working based on testing https://github.com/SynoCommunity/spksrc/pull/5095/commits/fbd98191653d2138140c697454d41e82edbd64ab
Confirmed, fixed https://github.com/SynoCommunity/spksrc/pull/5155
mercurial ✔️ ✔️ #5045
octoprint ✔️ ✔️ #5217
plexpy-custom ✔️ Ready for migration but BUG with the package install, to remove?
rdiff-backup ✔️ ✔️ #5447
rutorrent ✔️ ✔️ Migration to DSM7 #5455, #5669, #5617
sabnzbd ✔️ ✔️ ✔️ ✔️ #5109 (@Safihre)
#5137
#5527
sickchill ✔️ ✔️ #5431 @miigotu
salt-master ✔️ ✔️ #5017 co-author: @arnebjarne
salt-minion ✔️ ✔️ #5017 co-author: @arnebjarne, #5145
tvheadend ✔️ ✔️
znc ✔️ ✔️ #4951, #5091

Packages formally using python 2.x (#4524)

PACKAGE python 3.10 migration DSM7 migration Python 3.10 Published Python 3.11 migration Published 3.11 COMMENT
deluge ✔️ ✔️ ✔️ #5728 #4153, #5202, #5398
ffsync ✔️ ✔️ #5942
haproxy
salt-master ✔️ ✔️ ✔️ #5017 co-author: @arnebjarne

Framework clean-up

apps dependent packages Removed COMMENT
python3 (cross, spk, native) none ✔️ #5316
python38 (cross, spk, native) none TODO to remove once all python3.x packages published
cross/asyncstdlib none ✔️ #5175
cross/borgbackup borgbackup ✔️ #5048
cross/cryptography duplicity n/a Mandatory to build abi3 wheels
cross/cryptography-legacy python2 TODO to remove once all python3.x packages published
cross/deluge deluge ✔️ #5398
cross/dtlssocket homeassistant N/A Unable to migrate to pip
cross/duplicity duplicity ✔️ #5054
cross/gevent ffsync
cross/greenlet ffsync, python38
cross/immutables python3, python38 TODO to remove once all python3.x packages published
cross/libtorrent deluge ✔️ #5398
cross/lxml python2, python3, python38
cross/m2crypto python2
cross/mercurial mercurial ✔️ #5045
cross/msgpack-python python2, python3, python38
cross/numpy homeassistant ✔️ #5096, ARMv5 support pending: https://github.com/numpy/numpy/issues/20664
Fixed https://github.com/SynoCommunity/spksrc/pull/5155
cross/octoprint octoprint ✔️ #5217
cross/pillow none ✔️ #5175
cross/pillow_py2 python2
cross/pocketsphinx none ✔️ #5175
cross/poetry none ✔️ #5508
cross/pyalsa python2
cross/pyaudio python2
cross/pycryptodome none ✔️ #5175
cross/pycryptodomex none ✔️ #5175
cross/pycurl python2, python3, python38
cross/PyNaCl none ✔️ #5175
cross/pyyaml python2, python3, python38
cross/pyzmq python2
cross/rencode deluge ✔️ #5398
cross/ruamel.yaml python3, python38 TODO to remove once all python3.x packages published
cross/sphinxbase cross/pocketsphinx ✔️ #5175
cross/stockfish-niklasf spk/fishnet ✔️ @mayweed Rewritten in Rust (removal #5216)
cross/ujson homeassistant ✔️ #5096, Solved: https://github.com/ultrajson/ultrajson/issues/496
cross/umemcache none ✔️ #5175
cross/uwsgi python2
cross/webrtcvad none ✔️ #5175

th0ma7 avatar Jan 03 '22 17:01 th0ma7

@th0ma7 octoprint cannot install the plugins due to permission restrictions for the pip module/install folder. any suggestions?

2022-01-13 21:37:20,165 - octoprint.server - INFO - Listening on http://0.0.0.0:8088 and http://[::]:8088
2022-01-13 21:37:20,169 - octoprint.plugins.discovery - INFO - Registered OctoPrint instance on ds-arm for SSDP
2022-01-13 21:37:21,012 - octoprint.plugins.pluginmanager - INFO - Loaded plugin repository data from https://plugins.octoprint.org/plugins.json
2022-01-13 21:37:22,577 - octoprint.plugins.announcements - INFO - Loaded channel _important from https://octoprint.org/feeds/important.xml in 0.41s
2022-01-13 21:37:23,919 - octoprint.plugins.announcements - INFO - Loaded channel _releases from https://octoprint.org/feeds/releases.xml in 0.31s
2022-01-13 21:37:23,938 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-13 21:37:23,954 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> no), --user flag needed -> no, virtual env -> yes
2022-01-13 21:37:23,955 - octoprint.util.pip - INFO - ==> pip ok -> NO!
2022-01-13 21:37:23,955 - octoprint.util.pip - ERROR - Cannot use this pip install, can't write to the install dir and also can't use --user for installing. Check your setup and the permissions on /volume1/@appstore/octoprint/env/lib/python3.10/site-packages.
2022-01-13 21:37:23,989 - octoprint.plugin - ERROR - Error while calling plugin softwareupdate
Traceback (most recent call last):
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/plugin/__init__.py", line 271, in call_plugin
    result = getattr(plugin, method)(*args, **kwargs)
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/util/__init__.py", line 1737, in wrapper
    return f(*args, **kwargs)
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/plugins/softwareupdate/__init__.py", line 224, in on_after_startup
    self._check_environment()
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/plugins/softwareupdate/__init__.py", line 362, in _check_environment
    and get_comparable_version(versions["pip"])
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/util/version.py", line 170, in get_comparable_version
    version_string = normalize_version(version_string)
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/util/version.py", line 211, in normalize_version
    if "-" in version:
TypeError: argument of type 'NoneType' is not iterable

Edit:

  • DSM 6, aarch64-6.1
  • octoprint UI is running successfully

hgy59 avatar Jan 13 '22 21:01 hgy59

That's odd, never faced that issue. And good thing it isn't yet released, leaving time to settle on this.

th0ma7 avatar Jan 14 '22 01:01 th0ma7

Just a thought, I now wonder if the virtualenv didn't appeared to be installed by root user during the installation phase? Although I recall last time I tested homeassistant that many many other wheels where being installed after the fact with no problem. Perhaps something changed somehow...

I'll try to check this out over the week-end.

th0ma7 avatar Jan 14 '22 12:01 th0ma7

octoprint cannot install the plugins due to permission restrictions for the pip module/install folder. any suggestions?

I wasn't able to reproduce... I was able to successfully install Bed-Visualizer from the Install new Plugins... option. Although I noticed that a particular version of setuptools was YANKED on pypi, perhaps your install ended-up using it?

Installing plugin "Bed Level Visualizer" from https://github.com/jneilliii/OctoPrint-BedLevelVisualizer/archive/master.zip...
/volume1/@appstore/octoprint/env/bin/python3 -m pip --disable-pip-version-check install file:///tmp/tmpgfzttzb2/OctoPrint-BedLevelVisualizer-master.zip --no-cache-dir
Processing /tmp/tmpgfzttzb2/OctoPrint-BedLevelVisualizer-master.zip
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: OctoPrint in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages/OctoPrint-1.7.2-py3.10.egg (from Bed-Visualizer==1.1.1) (1.7.2)
Requirement already satisfied: OctoPrint-FileCheck>=2021.2.23 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.2.23)
Requirement already satisfied: OctoPrint-FirmwareCheck>=2021.10.11 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.10.11)
Requirement already satisfied: OctoPrint-PiSupport>=2021.10.28 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.10.28)
Requirement already satisfied: markupsafe<2.0,>=1.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.1)
Requirement already satisfied: markdown<3.2,>=3.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (3.1.1)
Requirement already satisfied: regex!=2018.11.6 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.10.23)
Requirement already satisfied: wrapt<1.13,>=1.12.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.12.1)
Requirement already satisfied: flask<2,>=1.1.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.4)
Requirement already satisfied: Jinja2<3,>=2.11.3 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.11.3)
Requirement already satisfied: Flask-Login<0.6,>=0.5 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.5.0)
Requirement already satisfied: Flask-Babel<2,>=1.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.0.0)
Requirement already satisfied: Flask-Assets<3,>=2.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.0)
Requirement already satisfied: werkzeug<2,>=1.0.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.0.1)
Requirement already satisfied: itsdangerous<2,>=1.1.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.0)
Requirement already satisfied: cachelib<0.2,>=0.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.1.1)
Requirement already satisfied: PyYAML<6,>=5.4.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (5.4.1)
Requirement already satisfied: pyserial<4,>=3.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (3.5)
Requirement already satisfied: netaddr<0.9,>=0.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.8.0)
Requirement already satisfied: watchdog==0.10.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.10.4)
Requirement already satisfied: sarge==0.1.6 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.1.6)
Requirement already satisfied: netifaces<1,>=0.11 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.11.0)
Requirement already satisfied: pylru<2,>=1.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.2.0)
Requirement already satisfied: pkginfo<2,>=1.7.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.7.1)
Requirement already satisfied: requests<3,>=2.26.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.26.0)
Requirement already satisfied: semantic_version<3,>=2.8.5 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.8.5)
Requirement already satisfied: psutil<6,>=5.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (5.8.0)
Requirement already satisfied: Click<8,>=7.1.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (7.1.2)
Requirement already satisfied: future<1,>=0.18.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.18.2)
Requirement already satisfied: websocket-client<1,>=0.59 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.59.0)
Requirement already satisfied: emoji<2,>=1.4.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.6.1)
Requirement already satisfied: sentry-sdk<2,>=1.3.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.4.3)
Requirement already satisfied: filetype<2,>=1.0.7 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.0.8)
Requirement already satisfied: zipstream-new<1.2,>=1.1.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.8)
Requirement already satisfied: blinker<2,>=1.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.4)
Requirement already satisfied: feedparser<7,>=6.0.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (6.0.8)
Requirement already satisfied: tornado<7,>=6 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (6.1)
Requirement already satisfied: zeroconf<0.34,>=0.33 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.33.4)
Requirement already satisfied: immutabledict<3,>=2.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.2.1)
Requirement already satisfied: pathvalidate<3,>=2.4.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.5.0)
Requirement already satisfied: colorlog<6,>=5.0.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (5.0.1)
Requirement already satisfied: unidecode in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.3.2)
Requirement already satisfied: pathtools>=0.1.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from watchdog==0.10.4->OctoPrint->Bed-Visualizer==1.1.1) (0.1.2)
Requirement already satisfied: sgmllib3k in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from feedparser<7,>=6.0.8->OctoPrint->Bed-Visualizer==1.1.1) (1.0.0)
Requirement already satisfied: webassets>=2.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from Flask-Assets<3,>=2.0->OctoPrint->Bed-Visualizer==1.1.1) (2.0)
Requirement already satisfied: Babel>=2.3 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from Flask-Babel<2,>=1.0->OctoPrint->Bed-Visualizer==1.1.1) (2.9.1)
Requirement already satisfied: pytz in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from Flask-Babel<2,>=1.0->OctoPrint->Bed-Visualizer==1.1.1) (2021.3)
Requirement already satisfied: setuptools>=36 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from markdown<3.2,>=3.1->OctoPrint->Bed-Visualizer==1.1.1) (60.5.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (2.0.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (1.26.8)
Requirement already satisfied: idna<4,>=2.5 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (2021.10.8)
Requirement already satisfied: six in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from websocket-client<1,>=0.59->OctoPrint->Bed-Visualizer==1.1.1) (1.16.0)
Requirement already satisfied: ifaddr>=0.1.7 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from zeroconf<0.34,>=0.33->OctoPrint->Bed-Visualizer==1.1.1) (0.1.7)
Building wheels for collected packages: Bed-Visualizer
Building wheel for Bed-Visualizer (setup.py): started
Building wheel for Bed-Visualizer (setup.py): finished with status 'done'
Created wheel for Bed-Visualizer: filename=Bed_Visualizer-1.1.1-py3-none-any.whl size=4182731 sha256=78edd206f54028eeb9218469443099aa8caaf5a231226448d2cf28619aeb095f
Stored in directory: /tmp/pip-ephem-wheel-cache-bul75zdr/wheels/b5/d6/23/c582f689507a9b7cf7420184f0fe2c5b595600fc122bac8917
Successfully built Bed-Visualizer
Installing collected packages: Bed-Visualizer
Successfully installed Bed-Visualizer-1.1.1
Done!

Would you mind uninstalling it / reinstalling it and see if the issue is still there? On my end I'll give it another shot using a fully clean tree on master to check if I can reproduce.

arch: kvmx64-7.0

EDIT: I'll also give it a shot on another arch such as armv7 / armv5 that I have available at home

EDIT2: Found the same log file as you referred, and all is good there as well (re-built from maser). Also still able to install an additional plugin:

2022-01-17 21:27:32,944 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-17 21:27:34,086 - octoprint.util.pip - INFO - Version of pip is 21.3.1
2022-01-17 21:27:34,088 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> yes), --user flag needed -> no, virtual env -> yes
2022-01-17 21:27:34,088 - octoprint.util.pip - INFO - ==> pip ok -> yes
2022-01-17 21:27:34,143 - octoprint.plugin.core - INFO - Initialized 17 plugin implementation(s)
2022-01-17 21:27:34,232 - octoprint.plugin.core - INFO - 17 plugin(s) registered with the system:

...
2022-01-17 21:31:35,501 - octoprint.plugins.pluginmanager - INFO - The plugin was installed successfully: Bed Visualizer, version 1.1.1
2022-01-17 21:31:35,893 - octoprint.plugins.tracking - INFO - Sent tracking event install_plugin, payload: {'plugin': 'bedlevelvisualizer', 'plugin_version': '1.1.1'}

EDIT 3: same on armv7, no issues:

2022-01-17 21:55:10,110 - octoprint.server.util.watchdog - INFO - ... initial scan done.
2022-01-17 21:55:11,037 - octoprint.plugins.discovery - INFO - Registered 'OctoPrint instance on DS115j-armv7._http._tcp.local.' for _http._tcp.local.
2022-01-17 21:55:12,074 - octoprint.plugins.discovery - INFO - Registered 'OctoPrint instance on DS115j-armv7._octoprint._tcp.local.' for _octoprint._tcp.local.
2022-01-17 21:55:12,086 - octoprint.plugins.discovery - INFO - Registered OctoPrint instance on DS115j-armv7 for SSDP
2022-01-17 21:55:12,106 - octoprint.server - INFO - Listening on http://0.0.0.0:8088 and http://[::]:8088
2022-01-17 21:55:13,905 - octoprint.plugins.pluginmanager - INFO - Loaded plugin repository data from https://plugins.octoprint.org/plugins.json
2022-01-17 21:55:22,503 - octoprint.plugins.announcements - INFO - Loaded channel _important from https://octoprint.org/feeds/important.xml in 1.5s
2022-01-17 21:55:29,709 - octoprint.plugins.announcements - INFO - Loaded channel _releases from https://octoprint.org/feeds/releases.xml in 2.4s
2022-01-17 21:55:33,037 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-17 21:55:33,082 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> yes), --user flag needed -> no, virtual env -> yes
2022-01-17 21:55:33,083 - octoprint.util.pip - INFO - ==> pip ok -> yes
2022-01-17 21:55:33,179 - octoprint.plugins.softwareupdate - INFO - Minimum free storage across all update relevant locations is 120.1MB. That is considered insufficient for updating.
2022-01-17 21:55:33,181 - octoprint.plugins.softwareupdate - INFO - Fetching check overlays from https://plugins.octoprint.org/update_check_overlay.json
2022-01-17 21:55:36,966 - octoprint.plugins.announcements - INFO - Loaded channel _blog from https://octoprint.org/feeds/octoblog.xml in 2.0s
2022-01-17 21:55:42,437 - octoprint.plugins.announcements - INFO - Loaded channel _plugins from https://plugins.octoprint.org/feed.xml in 1.4s
2022-01-17 21:55:47,198 - octoprint.plugins.announcements - INFO - Loaded channel _octopi from https://octoprint.org/feeds/octopi.xml in 1.2s
2022-01-17 21:55:48,419 - octoprint.plugins.pluginmanager - INFO - Loaded plugin notices data from https://plugins.octoprint.org/notices.json
2022-01-17 21:55:51,217 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-17 21:55:51,225 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> yes), --user flag needed -> no, virtual env -> yes
2022-01-17 21:55:51,226 - octoprint.util.pip - INFO - ==> pip ok -> yes
2022-01-17 21:55:51,354 - octoprint.plugins.softwareupdate - INFO - Saved version cache to disk

th0ma7 avatar Jan 18 '22 01:01 th0ma7

@miigotu the sickchill package is in theory ready for publishing. Was there any update planned that could benefit from being part of the python310 update?

@hgy59 on a similar note, I haven't published homeassistant just yet. Are you planning for an update anything soon?

th0ma7 avatar Mar 02 '22 19:03 th0ma7

@miigotu the sickchill package is in theory ready for publishing. Was there any update planned that could benefit from being part of the python310 update?

@hgy59 on a similar note, I haven't published homeassistant just yet. Are you planning for an update anything soon?

Sickchill should work fine from 3.6.2 to python 3.11.x, and possibly newer as long as all of the dependencies that require c-extenstion/rust-extensions are built.

miigotu avatar Mar 02 '22 20:03 miigotu

@hgy59 on a similar note, I haven't published homeassistant just yet. Are you planning for an update anything soon?

As I mentioned earlier, any homeassistant update needs cryptography ... that needs setuptools_rust to build the wheel. (I am currently on vacation and have no access to my spk dev env).

hgy59 avatar Mar 02 '22 21:03 hgy59

Thnx for your feedbacks to both of you. So it looks like rust is mandatory for any next steps...

th0ma7 avatar Mar 02 '22 21:03 th0ma7

Thnx for your feedbacks to both of you. So it looks like rust is mandatory for any next steps...

Yes, rust is going to become the defacto replacement for all c extensions over the next few years, and cryptography is a requirement of any application that makes web requests unless a pure python replacement comes along some day (crosses fingers). Cryptography could be done entirely in python with a small performance hit.

miigotu avatar Mar 02 '22 22:03 miigotu

rust is on my todo list but considering I've been sick over the last moth this will have to wait a little longer until I get better (or someone else tackle this over). In the meantime, we can stick with cryptography version 3.3.2 ...

th0ma7 avatar Mar 05 '22 13:03 th0ma7

@mayweed is there any interest in upgrading fishnet? Current version uses an older python3 and is 1.18.1 while new version is at 2.5.1. EDIT: Note that I can help in getting there but I have no clue how to use this software.

th0ma7 avatar Mar 26 '22 13:03 th0ma7

No, fishnet was rewritten in Rust actually. So it's a whole New package now...

mayweed avatar Mar 28 '22 09:03 mayweed

@mayweed So for now we can just abandon it until someone updates it? I don't think we do rust yet..

Safihre avatar Mar 28 '22 09:03 Safihre

No, fishnet was rewritten in Rust actually. So it's a whole New package now... @mayweed So for now we can just abandon it until someone updates it? I don't think we do rust yet..

I wonder if utilizing docker for pure rust builds is a good option for synocommunity?

FROM rust:1.59.0 AS builder
WORKDIR /fishnet
RUN mkdir /source
RUN git clone [email protected]:Pumalo/docker-conf.git /fichnet && (git submodule update --init --recursive || true) && cargo build --release -vv

FROM scratch as export-stage
RUN mkdir -p /output/$TARGETPLATFORM
COPY --from=builder /fishnet/fishnet /output/$TARGETPLATFORM/

name: Build all necessary architectures for fishnet

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Set up QEMU
      uses: docker/setup-qemu-action@v1
    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v1
    - name: Cache Docker layers
      uses: actions/cache@v2
      with:
        path: /tmp/.buildx-cache
        key: ${{ runner.os }}-buildx-${{ github.sha }}
        restore-keys: |
          ${{ runner.os }}-buildx-
    - name: Build docker image
      uses: docker/build-push-action@v2
      with:
        context: .
        platforms: linux/amd64,linux/arm64, <add all arch here>
        push: false
        output: type=local,dest=output_dir
        tags: fishnet-build
        cache-from: type=local,src=/tmp/.buildx-cache
        cache-to: type=local,dest=/tmp/.buildx-cache-new

You'd have all of the binaries for each platform in output_dir/$TARGETPLATFORM/fishnet, and then you add that to your spk however you want. I'm not sure how difficult it would be to offload some of the work to github actions in this way but it might be beneficial.

Maybe a third party repository to automatically build these binaries and add them as release artifacts, thatr synocommunnity can wget in the build process.

miigotu avatar Mar 30 '22 06:03 miigotu

Working on a POC just to see if I can build rust binaries that way: https://github.com/miigotu/fishnet-binaries

miigotu avatar Mar 30 '22 13:03 miigotu

Working on a POC just to see if I can build rust binaries that way: https://github.com/miigotu/fishnet-binaries

We already have rust support in spksrc. The following (cli) tools are built with rust:

  • cross/bat
  • cross/dua
  • cross/dutree
  • cross/exa
  • cross/ripgrep

hgy59 avatar Mar 30 '22 19:03 hgy59

Working on a POC just to see if I can build rust binaries that way: https://github.com/miigotu/fishnet-binaries

We already have rust support in spksrc. The following (cli) tools are built with rust:

  • cross/bat
  • cross/dua
  • cross/dutree
  • cross/exa
  • cross/ripgrep

I know that, but I still want to see if I can do this thing that way lol

miigotu avatar Mar 31 '22 02:03 miigotu

As I mentioned earlier, any homeassistant update needs cryptography ... that needs setuptools_rust to build the wheel.

@th0ma7 any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update 😄).

hgy59 avatar May 30 '22 05:05 hgy59

any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update 😄).

Nope, not yet. Nothing started neither.

th0ma7 avatar May 30 '22 06:05 th0ma7

Updated that python3 package was removed. Should we also remove/disable it on https://synocommunity.com/ ?

Safihre avatar Jun 18 '22 12:06 Safihre

Updated that python3 package was removed. Should we also remove/disable it on https://synocommunity.com/ ?

There are still packages that depend on python3 (i.e. 3.7) in the repo, that are not updated to Python 3.10/3.8 yet

  • plexpy-custom
  • rdiff-backup
  • rutorrent (?)

hgy59 avatar Jun 18 '22 20:06 hgy59

There are still packages that depend on python3 (i.e. 3.7) in the repo, that are not updated to Python 3.10/3.8 yet

  • plexpy-custom
  • rdiff-backup
  • rutorrent (?)

@hgy59 good catch, while they where all migrated to python310 but not published yet. And as for python38, only one left is domoticz.

th0ma7 avatar Jun 18 '22 21:06 th0ma7

any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update 😄).

Nope, not yet. Nothing started neither.

@th0ma7 may i ask if you already have a plan, when you will add rust support to enable cross compilation of newer cryptography versions? seems like everybody here thinks you are the right guy for this task. 🙏 😉

my target is to be able to build a wheel of cryptography 36.0.2, so that I can upgrade my homeassistant core installation using that wheel. but i am still a noob to spksrc cross-compilation toolchain, so i kind of depend on a working framework.

hogi1200 avatar Jul 05 '22 13:07 hogi1200

Sadly my cycles are really limited at the moment and will remain for a few weeks. Realistically this might slip until end of summer early fall unless i stumble on an easy solution before then.

th0ma7 avatar Jul 05 '22 16:07 th0ma7

@th0ma7 thanks for the quick response. Your contributions are still highly appreciated 👍

hogi1200 avatar Jul 06 '22 10:07 hogi1200

@th0ma7 did you ever consider gateone package? (https://github.com/SynoCommunity/spksrc/issues/4524#issuecomment-1213032217)

hgy59 avatar Aug 12 '22 18:08 hgy59

@th0ma7 did you ever consider gateone package? (https://github.com/SynoCommunity/spksrc/issues/4524#issuecomment-1213032217)

Nope, and sadly no clue what taht is (on vacation with limited internet)

th0ma7 avatar Aug 13 '22 00:08 th0ma7

@th0ma7 did you ever consider gateone package? (https://github.com/SynoCommunity/spksrc/issues/4524#issuecomment-1213032217)

Nope, and sadly no clue what taht is (on vacation with limited internet)

"Gate One is an HTML5-powered terminal emulator and SSH client"

https://github.com/liftoff/GateOne

miigotu avatar Aug 13 '22 03:08 miigotu

@cytec I tested the plexpy-custom package which makes use github.com/Tautulli/Tautulli.git Is there an interest at keeping this package alive and maintained? Thnx in advance.

th0ma7 avatar Oct 13 '22 01:10 th0ma7

any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update smile).

Nope, not yet. Nothing started neither.

@th0ma7 may i ask if you already have a plan, when you will add rust support to enable cross compilation of newer cryptography versions? seems like everybody here thinks you are the right guy for this task. pray wink

my target is to be able to build a wheel of cryptography 36.0.2, so that I can upgrade my homeassistant core installation using that wheel. but i am still a noob to spksrc cross-compilation toolchain, so i kind of depend on a working framework.

@hogi1200 rust wheels is now available on master, enjoy :)

th0ma7 avatar Oct 13 '22 01:10 th0ma7