Better Klipper support / Extension repository

[feelfreelinux]

• [x] basic klipper support with OctoKlipper addon
• [x] basic klipper documentation, install script
• [ ] Fluidd / moonraker / mainsail support
• [ ] Replace script-based installation with extension repository option in the app

[melon-husk]

There is a issue related to filesystem when I tried to install moonraker, it throws some db error

[marcfon]

There is a issue related to filesystem when I tried to install moonraker, it throws some db error

@melon-husk could you share how you (tried to) install moonraker? I'm also on that journey.

[melon-husk]

Search this on youtube, klipper on android 실험실 , you'll see a korean guy install klipper, moonraker and mainsail on android using tmux. Those scripts won't work directly on octo4a cause it uses alpine linux so you'd have to look through this one and this one as well to attempt to install it. I already found a solution to run klipper, moonraker and fluidd on android, you can add me on discord if you want to know how I did it. My discord tag is Melon Husk#0505

[jindaxia]

There is a issue related to filesystem when I tried to install moonraker, it throws some db error

Same issue with you, I solved the dependences of moonraker and run, but it seems that the lmdb can not running on the tmpfs filesystem.

[melon-husk]

There is a issue related to filesystem when I tried to install moonraker, it throws some db error

Same issue with you, I solved the dependences of moonraker and run, but it seems that the lmdb can not running on the tmpfs filesystem.

yup, I tried to debug it with strace but no luck, it makes some syscall that is not supported I think. But I found a way to get klipper, mainsail and moonraker work.

[jindaxia]

There is a issue related to filesystem when I tried to install moonraker, it throws some db error

Same issue with you, I solved the dependences of moonraker and run, but it seems that the lmdb can not running on the tmpfs filesystem.

yup, I tried to debug it with strace but no luck, it makes some syscall that is not supported I think. But I found a way to get klipper, mainsail and moonraker work.

can you show me your code , or just told me your main idea?

[thingsapart]

I think I may have overcome this particular problem after a few days of fighting with it; even trying, compiling and installing different proot implementations in octo4a bootstrap as Moonraker seems to work under termux but I didn't really want to run several "distros".

I don't have the whole thing running end-to-end including klipper yet, so maybe there'll be other problems, but at least Moonraker gets past database init and runs. I get occassional Error running shell command: '['ip', '-json', 'address']' errors but it's not fatal.

It looks like locking is the problem, applying this patch (pass lock=False to lmdb.open(...)) does it for me:

diff --git a/moonraker/components/database.py b/moonraker/components/database.py
index 4ee3444..06391d8 100644
--- a/moonraker/components/database.py
+++ b/moonraker/components/database.py
@@ -83,7 +83,7 @@ class MoonrakerDatabase:
         if not os.path.isdir(self.database_path):
             os.mkdir(self.database_path)
         self.lmdb_env = lmdb.open(self.database_path, map_size=MAX_DB_SIZE,
-                                  max_dbs=MAX_NAMESPACES)
+                                  max_dbs=MAX_NAMESPACES, lock=False)
         with self.lmdb_env.begin(write=True, buffers=True) as txn:
             # lookup existing namespaces
             with txn.cursor() as cursor:

Oh, and this might cause huge problems when accidentally running multiple instances on the same db files.

[RyanEwen]

I've been trying to run Klipper + Moonraker + Mainsail from an Android tablet and the one thing holding me back was the lack of driver support for CH34x on my tablet. When I found out that Octo4a came with its own driver I decided to try installing everything within the Octo4a environment, but it was tough. I'm happy to have found this extension and script to run Klipper through the Octo4a environment, but it seems pointless without Moonraker as well.

The cleanest setup I've managed to find so far is Linux Deploy with a Debian chroot container - but without the driver, Klipper cannot connect. So what I am trying now is Octo4a + Klipper, and Linux Deploy + Moonraker + Mainsail. The problem with the latter is that Klipper only exposes a Unix Domain Socket at /tmp/klippy_uds which cannot be accessed by my Debian container within Linux Deploy.

My question is, is there a way to pass in a custom mount to Octo4a's Alpine, so that Klipper can use that for the Unix Domain socket, and Moonraker within Debian within Linux Deploy can also mount it and access it, to talk to each other?

[RyanEwen]

Or if you have any ideas on how to reuse the work done to bring the CH34x driver from Octo4a out to Linux Deploy (or Termux), that would be amazing

[thingsapart]

I'm not 100% familiar but I think I pieced it together - and it's quite ingenious! Octo4a creates a files in /dev ("/dev/ttyOcto4a") that e.g. octoPrint and Klipper can read/write from and to [1], kind of like a normal serial port. Except that file doesn't support the necessary ioctl functions to manipulate a pty/tty. And that's why octoprint also provides another file [2] with which to control the pty - but it then needs to intercept these stdlib ioctl calls via LD_PRELOAD-ing an intercept library [3] and directly communicating with the Java driver instead.

BootstrapRepository.kt shows some of this logic of launching the alpine bootstrapped proot [4] around line 177-190.

I'm sure it could be ported to termux via an extension or LinuxDeploy (and Termux for whatever reason runs moonraker without the locking problems that octo4a's alpine bootstrap seems to have) but it would seem to take a lot of work.

[1] https://github.com/feelfreelinux/octo4a/blob/master/app/app/src/main/java/com/octo4a/serial/VirtualSerialDriver.kt [2] /home/octoprint/eventPipe => https://github.com/feelfreelinux/octo4a/blob/master/app/app/src/main/cpp/vsp-pty.cpp [3] https://github.com/feelfreelinux/octo4a/blob/master/app/app/src/main/cpp/ioctl-hook.c [4] https://github.com/feelfreelinux/octo4a/blob/master/app/app/src/main/java/com/octo4a/repository/BootstrapRepository.kt

[RyanEwen]

As you were writing that I was noticing the proot bind: -b /data/data/com.octo4a/files/serialpipe:/dev/ttyOcto4a

I found that /data/data/com.octo4a/files/serialpipe was a symlink to /dev/pts/0. Which means that /dev/ttyOcto4a within Alpine might be /dev/pts/0 outside, in Android.

In Linux Deploy / Debian, I setup printer.cfg to use serial: /dev/pts/0 and started up Klipper. It worked! Then I started Moonraker and nginx (for Mainsail), and it's all working!

So to sum it up:

• I have Octo4a installed and running. No scripts or commands or anything else needed there.
• I have Linux Deploy with Debian installed and running. I used KIAUH to install Klipper + Moonraker + Mainsail within that. I configured printer.cfg to use serial: /dev/pts/0.

Now I just need to figure out how to cleanly automate the startup of Klipper + Moonraker + Mainsail. Linux Deploy supports some init systems. I just don't know much about that stuff yet.

[thingsapart]

Awesome! I think Linux Deploy implies that the phone is rooted, right? Afaik, unrooted phones might not have access to /dev/pts/0 and therefore the workarounds.

[RyanEwen]

Correct - I rooted my tablet for this. I also had to sudo chmod 777 /dev/pts/0 within Debian

[RyanEwen]

It would be amazing if we could swap out Alpine for Debian here in Octo4a. Then it could probably run Klipper + Moonraker + Mainsail no problem

[thingsapart]

I tried faking an init system by writing a simple (bad) systemctl shell-script wrapper with "[machine] provider: systemd_cli" in the klipper cfg. That seems to allow me to at least restart klipper from within mainsail but i still need to ssh in to start the processes in the first place.

[thingsapart]

It would be amazing if we could swap out Alpine for Debian here in Octo4a. Then it could probably run Klipper + Moonraker + Mainsail no problem

I am actually running Klipper + Moonraker + Mainsail within Octo4a's Alpine, but moonraker needs a one-line patch for how it opens LMDB. This might be risky since it won't lock the DB -- but afaict the DB isn't that vital and might be safe if you're only running a single instance anyway.

The issue might be that Octo4a's proot or something about how it is run/set up doesn't support some of the locking syscalls afaict. I tried compiling my own proot (no luck) and running termux's proot in Octo4a but had the same problem, so maybe it's something else after all. I don't think it's Alpine vs Debian, though debian would probably support systemd init without hacks.

[RyanEwen]

How difficult was it to install Moonraker and Mainsail within Octo4a's Alpine? And to apply that patch you mentioned?

[thingsapart]

Not super difficult - I took some rough notes of the commands I ran and I still have my config files and such -- I think I can probably turn that into a PR and install-script for Octo4a or at least a self-hosted script in a few hours.

I mainly just followed the manual installation steps for Moonraker and Mainsail and found the necessary package dependencies along the way. The moonraker patch is a couple posts above (just adding , lock=False in database.py's lmdb.open(... call).

I uploaded my stream-of-consciousness type installation notes (linked below), they're unfortunately complicated by also installing Python 3.11 at the same time:

I think the best approach is to open the official guide to installing Klipper, Moonraker and Mainsail and follow that and look at my notes to see which apk packages need to be installed and maybe differing commands be run.

https://github.com/thingsapart/octo4a/blob/thingsapart-klipper-install/klipper_command_notes.md

But I'd like to put all this into a single script some time soon-ish.

[greenbigfrog]

Setup a script for moonraker following @thingsapart 's notes. Went quite smooth, although I haven't been able to IRL test yet, since I'm just getting my printer tomorrow.

You can find it at https://github.com/greenbigfrog/octo4a/blob/klipper-improvements/scripts/setup-moonraker.sh Gonna clean it up a bit and properly IRL test it sometime before opening a PR. Also gonna try my hand at getting mainsail running.

To avoid problems with multiple instances accessing the same db, perhaps call the kill script at the beginning of the start script?

[RyanEwen]

In case it helps anyone I also documented my adventure. Here's how I got everything going in Debian via Linux Deploy on a rooted device:

https://gist.github.com/RyanEwen/ae81fc48ad00397f1026915f0e6beed9

[greenbigfrog]

Wow, what a ~~wasted~~ day. Spent most of it wondering why the fuck I wasn't able to get any communication. At some point I simply uninstalled everything, since I just couldn't get it to work. I then realized that I have to activate OTG before use on my 3t each time... It isn't necessary on my 6t... Well, but I got it working now...

~~For klipper I needed the LD_PRELOAD=/home/octoprint/ioctlHook.so, but just octoprint was able to communicate without it~~ Nvm. It's probably just loaded by octoprint already...

Got some more or less working scripts pushed now, but still very rough and not plug and play.

One question that came up was: Should dependencies be duplicated between extensions so they can be installed in any order? If yes, then the plugin extras thingie definitely should be removed and just duplicated/split across the others.

[strahlermann]

Thank you so much @greenbigfrog scripts are working fine, even if not Plug & Play would love to have this working in just some clicks, trying to get Klipperscreen working atm.

[thingsapart]

It's been a while, but I have a first working version of a fork of Octo4a that installs Klipper + Moonraker + Mainsail automatically. I've been working on this for about two weeks now most of my spare time. I've had lots of trouble getting the bootstrapped environment just right - lots of programs like dpkg, proot or even the shell do not work if everything is not "just right". Still has some problems and minor warts but I got Klipper+Moonraker+Mainsail running (though I haven't tested connecting to a printer yet).

The bootstrap is now a termux-proot-distro image based on Ubuntu (could just as well be Debian or other derivative of it) which allows calling into Kiauh to install packages and configurations. One downside is that a full installation with all of Klipper/Moonraker's dependencies is taking up about 4.5GB(!).

The branch is here: https://github.com/thingsapart/octo4a/tree/ubuntu-klipper-rebase

I'll probably keep working on this. I'm sooo tired of dealing with Linux-based SBCs, SD card images and barely supported webcams and touchscreens. Phones for controlling 3D printers have their own sets of problems but I currently prefer them to SBCs.

My LG G6 with the original Octo4a/Alpine-based Klipper installation from Aug 10 now has around 100 hours of print time on it and I've only had one problem where Octo4a died and I never even reboot the phone. I'm not sure if it was the OS running out of memory or the app/bootstrap/Klipper legitimately crashing. Either way, pretty happy with that.

Btw, the more I dive into Octo4a, the more I admire all the work and ingenuity it took to pull it off! Amazing!

[SkyShadex]

Just gave it a go with your alpha and it seems like it tapped got to 100% but in looking at the log, something went south near the end with the proot.

Gonna give it another go.

Moto x pure edition.

[greenbigfrog]

Note to self (and others in case I don't get around to updating my fork): Missing packages libsodium iproute2

Got some motivation again today, so I hardwired a PSU to a phone, swapped the board in the printer, and ran into a few issues/TODOs.

One that's got me stumbling, is why my system refuses to let moonraker create a socket at the default location of '/root/printer_data/comms/moonraker.sock', but using some other socket works just fine. Even this KISS script errors for that location. No idea why

import os
import socket
path = os.fspath('/root/printer_data/comms/moonraker.sock')
sock = socket.socket(1, 1)
sock.bind(path)

[aleung]

@greenbigfrog I got the same error after upgrading moonraker. I just change the sock path to /tmp then it works fine:

moonraker/components/extensions.py:

    # sock_path = comms_path.joinpath("moonraker.sock")
    sock_path = "/tmp/moonraker.sock"