Can we add Unison and Duplicity?

[gerroon]

Hi

It would be great if we have Unison and Duplicity in the repo. This will allow good backup options on Android.

[justincase2006]

Yes

[fornwall]

The package suggestions have been added to https://termux.com/package-suggestions.html

[Neo-Oli]

Hi, I am reopening this issue. Package requests will now be labeled and tracked as an issue.

[ghost]

+1 for Unison.

[tristan-k]

+1 for Unison

[ghost]

+1 for unison

[ghost]

Just to help a bit

Jens Wagner has compiled a static ARM binary for unison 2.40.102. He is hosting it, as well as instruction for compilation, on his web site http://www.crutzi.info/unison/binary/armel.

[sunziping2016]

+1 for unison

[SapphicCode]

+1 for Unison

[vovcacik]

I spend whole day on getting unison into my android phone. I've got some notes more or less relevant to this issue.

Getting debugging toolchain for Android:

The SDK Manager and USB driver

• for the USB driver get SDK Manager
• run sdkmanager.bat extras;google;usb_driver

The adb.exe and fastboot.exe

• you can get just the debug tools from SDK Platform Tools, without the unnecessary stuff
• list devices with adb devices -l
• get shell with adb shell

Unison binaries

Linux repositories

• it might have been stupid idea, but don't bother with unison binaries from
  • Arch linux for ARM (I tried arm and arm7h platforms)
  • Unison for Android app (it features 9 versions of unison, including stable 2.48.4). Unpacking this app gave me non-executable binary files like unison-armeabi-2.48.4.enc. By the extension I think it is encrypted or UUencoded.
  • TODO it might be worth trying Debian ARM package repo

Crutzi build

• the only one that worked for me, unfortunately the version is old
• his build seems to be the easiest to reproduce
  • OCaml does support Linux on ARM, but not crosscompilation to it (there are patches that might make it possible), you might want to build natively instead as follows:
  • there are QEMU prebuild images for ARM arch here

Running Unison in Android

• for some reason my ROM requires the binary to be located in /system/bin; I believe it is permission problem - I've got the basic ACL right, but SELinux was probably the culprit
• you can easily alter your ROM with mount -o remount,rw /system. If the device is busy, go to recovery environment and do it from there - the actual command will look more like this: mount -t ext4 -o rw,noatime,data=ordered /dev/block/platform/msm_sdcc.1/by-name/system /rom (to get parameters relevant to you run mount | grep /system from OS). Make sure to read full howto.
• running unpriviledged unison in my ROM produced quite a few problems:
  • PATH env var: because /sdcard/ is mounted with noexec and because other locations like /data/local wont let me execute binary (SELinux...) I had to put it in /system/bin, see beginning of this section
  • broken su --preserve-environment -c ...: I tried four ssh servers, but only one worked (see section below)
  • overriding HOME env var for Unison: my ROM had HOME environmental variable set to HOME=/ which is readonly. I solved this with changing the ROM: a) /system/etc/mkshrc and b) /system/etc/profile. The latter may be unnecessary when you have properly working su --preserve-environment -c .... For details see this. Also this can be probably simplified with termux, because it provides OpenSSH (and not just Dropbear as other apps) using ~/.ssh/environment; not sure though. Other OpenSSH options include SendEnv/AcceptEnv.

broken su --preserve-environment -c ...

• this whole section is about simple problem:
  • SuperSu is not preserving environmental variables for direct commands over ssh, unless you specify --preserve-environment parameter. For direct commands versus interactive ssh session see section below. Unison is using direct commands when you are using ssh method (see unison docs for ssh method vs socket method). With this setup you need to run unison as su --preserve-environment -c unison which preserves proper $HOME env var. However some ssh servers will interpret this command as interactive session, and hence is equivalent to su --preserve-environment -l, which wont work at all.
• the combination of SuperSu app (it provides /su/bin/su) and 3 ssh servers: Servers Ultimate, SSHDroid and SimpleSSHD had this problem (I think all three are using Dropbear)
• only the combination of termux which provides both the $PREFIX/bin/su and $PREFIX/bin/sshd (OpenSSH) worked for me. SuperSu and termux sshd works fine together.

Direct ssh commands vs interactive ssh sessions

• suppose you have configured your environmental variables in ~/.profile and/or ~/.bashrc or ~/.mkshrc etc.
• when you are using interactive session (ssh admin@host or putty...) all is fine, your shell will find and execute the rc files
• when you are using non-interactive session commands are sent directly (plink admin@host "unison -server for example) and thats exactly what unison does - the environment may get resetted if you configured the ssh server to use su shell for clients or you sent command like plink admin@host "su --preserve-environment -c unison -server. This is bad when the unison on server depends on properly set $HOME variable

Unison caveats

• unison is sensitive about versions - you should run at least the same MAJOR.MINOR, even better MAJOR.MINOR.PATCH (definition).
• unison is sensitive about what version of OCaml you used during compilation even with same sources
  • due to some internal changes in OCaml, unison may throw Uncaught exception Failure("input_value: bad bigarray kind"). Google or here.
  • another example is easy to find in official windows builds

[vovcacik]

Android binaries in https://github.com/vovcacik/unison-build-scripts. Can anyone package it?

[vovcacik]

Unison usage notes for Android here https://github.com/vovcacik/unison-build-scripts/blob/master/ANDROID.md

[YtvwlD]

duplicity is already installable:

• Install gnupg, librsync-dev, clang, python2 and python2-dev via apt.
• Install duplicity via pip2.
• Edit ~/usr/bin/duplicity with your favourite text editor and set the shebang to the output of which python2.

(This is just a fancy "me too".)

[niklasl]

Until Unison becomes available, I use a crude fallback method:

• rsync local files to a "landing directory" on the server;
• use unison to consolidate the landing dir with the real sync dir locally on the server;
• rsync back the results.

Not sure about how clever that is, but perhaps it'll help someone.

[gerroon]

@niklasl you can also install syncthing in termux. It was added recently. It is a better syncing solution then bi directional rysnc.

I use rsync to back my whole sd card btw.

[YtvwlD]

You can also install Syncthing as a normal Android app.

[gerroon]

@YtvwlD

I do not recommend the regular syncthing android app, it is super buggy and crashy especially with large shares. The app regularly crashes the syncthing binary and you will never know about it.

Running it under termux much more flexible.

[niklasl]

@gerroon Thanks for the tip! It seems syncthing is configurable in a different manner than Unison, but it seems like an interesting alternative.

[SDRausty]

It appears that both Duplicity and Unison are currently available in Termux through PRoot Arch Linux. This script is designed to install Arch Linux in just a few keystrokes. Tapping this link https://sdrausty.github.io/TermuxArch/setupTermuxArch.sh will transfer setupTermuxArch.sh to your device. Simply run bash ~/storage/downloads/setupTermuxArch.sh to install Arch Linux in Termux. To install Duplicity use pacman -S duplicity. Running pacman -S unison will install Unison on device.

[markbil]

@sdrausty i like your idea. I installed the unison package in termuxarch via "pacman -S unison". now i would like to ssh from my pc (in the same local network) directly into termuxarch to sync files via unison. however "ssh -p 8022 [email protected]>" connects me to the termux environment not termuxarch, so unison commands in termuxarch are not available directly via ssh (e.g. "ssh -p 8022 [email protected] unison -version" results in "command not found"). "ssh -p 22 [email protected]" results into "connection refused" error, even though I have set up an open ssh server in termuxarch. any ideas how I can ssh directly into termuxarch?

[SDRausty]

@markbil I am glad to hear you like the idea. Use a higher numbered port, something like 8222 in TermuxArch for instance.

[alexozer]

I've tried Unison in Arch, but when using -repeat watch to sync instantly on local file changes, after a while of running it seems to stop syncing changes on the phone eventually.

At some point I'll modify a file, and unison will print "Looking for changes" and then just freeze until restarted, when it will then sync the change. No idea how to proceed.

[vovcacik]

@alexozer that's off topic for this thread. I've got some ideas, start new issue so we can discuss it there.

EDIT: see #2412

[alexozer]

@vovcacik Sorry about that, don't know why I thought it was relevant to this thread at the time.

[QGB]

"termux"+ setfacl getfacl

[natethern]

@niklasl I was doing the same thing as you (rsync, then unison, then rsync back). Then I switched to using sshfs to create a local mount of the directory (I don't know if that can be done on windoze - I use debian, among others). My process when I come home is (was)

1. on android: open termux and start sshd
2. on linux: run sshfs ... to mount the directory on my android device
3. on linux: run unison ... to sync
4. on linux: run fusermount -u ... to unmount

I also wrote it up as an bash script. As an example, for my one 3-way sync the script does this:

1. Initiate the sshfs mount on device A
2. Initiate the sshfs mount on device B
3. Sync device A with the master using unison
4. Sync device B with the master using unison (propagates any updates from A into B)
5. Sync device A with the master using unison (propagates any updates from B into A)
6. unmount A & B

The script checks for empty directories at the mount points (meaning the mount failed) and skips unison if appropriate.

Recently I switched to SimpleSSHD which automatically keeps an sshd server alive on each of my android devices. This, of course, is a non-termux solution.

[621625]

+1 for Duplicity.

Please just add support for Duplicity, it would be so cool.

[hellas2000]

I'm going to buy an android phone, but I'm waiting for unison to work in termux, it's very important to me. Please let an expert to help us :)

[tulakalbeyo]

+1 for unison

[SDRausty]

@tulakalbeyo unison is available in Termux through TermuxArch:

$ startarch s user yay unison
Warning: your password will expire in 0 days.
8 aur/unison-232-compat 2.32.52-2 (+0 0.00) (Orphaned)
    File-synchronization tool (legacy 2.32 version). Compatible with parallel installation to other versions of unison binaries (but pay attention to the local user configurations!)
7 aur/unison-git 2.51.3.r257.g22a05df-1 (+1 0.00)
    File-synchronization tool
6 aur/csync2-git 2.0.r22.gce67c55-1 (+1 0.00)
    Asynchronous cluster syncronisation tool based on librsync and inspired by Unison
5 aur/ucm-bin M1m-1 (+2 0.39)
    Unison language code manager
4 aur/unison-240-compat 2.40.128-1 (+3 0.00) (Orphaned)
    File-synchronization tool
3 aur/csync2 2.0-4 (+10 0.00)
    Asynchronous cluster syncronisation tool based on librsync and inspired by Unison
2 aur/soundfont-unison 1.00-3 (+32 0.00)
    A lean and clean GM/GS soundbank
1 extra/unison 2.51.2-9 (1.9 MiB 8.8 MiB)
    File-synchronization tool
==> Packages to install (eg: 1 2 3, 1-3 or ^4)

going to buy an android phone, but I'm waiting for unison to work in termux

[user15:30~]$ unison
Usage: unison [options]
    or unison root1 root2 [options]
    or unison profilename [options]

For a list of options, type "unison -help".
For a tutorial on basic usage, type "unison -doc tutorial".
For other documentation, type "unison -doc topics".

just add support for duplicity, it would be so cool

[user15:34~]$ duplicity -V
duplicity 0.8.15

@bitcoinmeetups both give the impression of working in Termux, and they do need proot.