bittorrent support ?

[eoli3n]

Hi,

I work in a University. 5 years ago we searched for all deployment systems and we did'nt find any project which fits our needs. We used sometimes clonezilla but in unicast, for 800 nodes... We tried DRBL, but our nodes are in A Class, so very poor performance, and install documentations are.... you know :)

So i made my own solution, in bash, based on murder from twitter : https://github.com/lg/murder It's now working well, i can deploy my GNU/Linux image (4,5G xz -9e compressed) on 800 nodes in 45min max, using nfsroot instead of squashfs.

I will probably start to make the project more distribuable, in python instead of bash. The problem is that my solution is a clonezilla-like, which could probably share lots of code (dialog menus for example).

I saw that there are some commits about bttrack|bittorrent etc... Can you tell us a little more ? When will this be tryable in unstable version ?

Thanks, have a nice day

[stevenshiau]

Thanks for asking. It's still in early development status, and still, it is used with DRBL. :) We had some tests here but the steps are still very primitive, therefore not open for tests yet. As for when, well, maybe in a few months we can release a testing version for that. BTW, could you please share your solution based on murder? It might help us to improve our developing bittorrent solution. Thanks.

Steven

[eoli3n]

I will, but i can't as is for now. I just use murder (which only is bittornado with some custom configurations to speed up transfert), to start a tracker, create a torrent file, launch seeds and peers.

Please look at https://vimeo.com/11280885

Have a nice day

[stevenshiau]

OK, thanks. However, for bare metal recovery, there is no good way to find the temp working dir for the downloaded image, especially when the image is huge, e.g., 10 GB. Normal machine does not have such big RAM so ramdisk won't work. Of course, it's possible to split the image then use BT to share that. However, this mechanism will be complicated. I am wondering how you put the downloaded image in your BT client?

Steven

[eoli3n]

That was my problematic too... I resolved it by storing partition's archive on the destination partition, temporary. (tar | xz packages)

But that's not a good solution, because it limits you to deploy images where partition_compressed_package_size < ( partition_size - raw_datas_size ), and you cannot predict partition_compressed_package_size.

So a full filesystem (100% used) packaged cannot be restored on a identical partition with that solution.

What's i'm working on, is a modulable bt deploy solution, which requires user to write some "playbooks", complete deploy | image_create scenarios using modules, probably based on ansible, avoiding dialog menus, more declarative. And then, user need to choose himself where to put the file, adapted to his partition table.

Create a package

package :
  _package_name_
partition_table :
  dump
savefs :
  sda2, xz, default
  sda3, xz, default
  sda5, gzip, extra
bittorrent :
  create_file, default
  create_file, extra

Restore a package

package :
  _package_name_
partition_table :
  restore
erase : 
  sda2
  sda3
bittorrent :
  peer, default, sda5 #temp storage partition
  peer, extra, sda2 #temp storage partition
restfs :
  sda2, peered default #it knows then where to get it
  sda3, peered default
  sda5, peered extra
shell : rm somedir/somefile
bootloader :
  grub, install, update

[don-rumata]

Bittorrent filesystem based on FUSE: https://github.com/johang/btfs

[tjjh89017]

@eoli3n You can check out EZIO for BT-based. Ezio can write the data to disk directly. Ezio = BT Client + Raw Disk Writer But we still work on the docs and the usage.

In a few month, ezio will be a part of clonezilla

[eoli3n]

hm, raw disk... that's bt + dd ? We have 250G HDD to write, even with bittorrent... But i will take a look, good to know

[tjjh89017]

@eoli3n in first release, Ezio = bt + dd but in latest version, we can read/write/transfer only the "used" block in the filesystem. e.g. a 250GiB disk, only use 10GiB We just distribute 10GiB data via BT

[eoli3n]

wow, need to try..!