csync2
csync2 copied to clipboard
LINBIT
csync2 still actively developped or "only" maintained ?
The title says it all ;-) I am actually asking because I am looking forward for more options in the auto-conflic-resolution part (eg. "auto" statement).
It would be great to have an additional mode specifying the winning node directly (ex.: "auto host host1") specifying which host will win (this may of course require an additional run (possibly automatic) in multi-host configs.
Also every "auto" statement should allow for a file pattern (just like exclude) applying it to only those files. (if not present changes the global setting, thus not breaking any existing config)
On Sat, 9 Jan 2021 at 14:05, ggs67 [email protected] wrote:
The title says it all ;-) I am actually asking because I am looking forward for more options in the auto-conflic-resolution part (eg. "auto" statement).
It would be great to have an additional mode specifying the winning node directly (ex.: "auto host host1") specifying which host will win (this may of course require an additional run (possibly automatic) in multi-host configs.
Also every "auto" statement should allow for a file pattern (just like exclude) applying it to only those files. (if not present changes the global setting, thus not breaking any existing config)
"Patches accepted" :-)
Ok, I will think about it. Unfortunately my time is very limited as by work and other projects. But I could well consider working on this. I will fork and give it a try, Esspecially since I stumbled over some other features I would like to see, like dynamic auto-conflict resolution.
Will you consider merging patches from schafdog csync2 fork. Some nice extra features there.
https://github.com/schafdog/csync2
On Mon, Jan 18, 2021 at 01:09:53AM -0800, stfast wrote:
Will you consider merging patches from schafdog csync2 fork. Some nice extra features there. https://github.com/schafdog/csync2
I should find some time for reviewing/merging stuff in early Feb. Feel free to reminde me ;-)
From reading the README of the @schafdog csync2 fork, this seems to have been forked 10 years ago and feels more like a rewrite than something that can be easily "merged".
Given that csync2 mostly works for us as it is, it is unlikely that I spend much effort "actively" trying to merge things, but if "someone" would prepare a number of "digestible" pull requests, that would be welcome.
Lars
Even though I personally would have designed csync2 differently, possibly even as an extension to rsync rather than something stand-alone. But someone would "just" have to do that as well.
I will attempt merging some of the stuff such as code cleanup, less FS scans and less connections to make it work faster , using less resources. I have made a pull request with some fixes to current major problems.
There is also a "new" issue since there is xinetd package (which triggers csync2) and also csync2 package itself removed from RedHat Fedora 34+ distribution. Obvious reason for this removal is xinetd dependency. Did you consider running csync2 as a daemon (via csync2d)?
Now, when I know that such a problem will be coming - will see options on how to solve that. I think I saw somewhere Csync2 docs mentioned as it is being able to run as a daemon but cannot find clear instructions on how such installation is made/run. Would you point me in the right direction?
Now, when I know that such a problem will be coming - will see options on how to solve that. I think I saw somewhere Csync2 docs mentioned as it is being able to run as a daemon but cannot find clear instructions on how such installation is made/run. Would you point me in the right direction?
-ii
No, it does not run for me in that mode, gives:
csync2 -ii Server error: Address already in use
Will have a look at this later.
Yep... Because you already have a process running on port 30865 and this is most likely another csync2 instance over inetd
# ss -lnp |grep 30865
tcp LISTEN 0 128 *:30865 *:* users:(("inetd",pid=515,fd=7))
@stfast the only thing the distribution needs to do is run csync2 -ii. My opinion is that creating start scripts should be created by the distributions themselves, but if possible, you can add systemd unit files and init.d to csync2 project
Yes, there was another process on that port. Not another csync2 instance but reserved by Xinetd for csync2 itself (which is not running at the moment). Would have to disable it to run csync2 -ii however - have not such need now.
Indeed, you have to stop xinetd:
$ csync2 -ii Server error: Address already in use
$ systemctl stop xinetd
$ csync2 -vvvvv -ii Listening on 0.0.0.0:30865 as server.domain.com. Csync2 daemon running. Waiting for connections.
$ systemctl start xinetd
$ csync2 -ii Server error: Address already in use
Actualy there is allready systemd csync2.service file
https://github.com/LINBIT/csync2/blob/master/csync2%40.service
but should be improved a little.
I use csync2 last git with fedora 34 without xinetd since 6 months now without prob. here is the units:
csync2.socket
[Socket] ListenStream=30865 Accept=yes
[Install] WantedBy=sockets.target
[Unit] Description=csync2 connection handler After=syslog.target
[Service] ExecStart=-/usr/sbin/csync2 -i -v StandardInput=socket StandardOutput=socket
I compiled csync2 with these options ./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc/csync2
I have moved away from Xinetd use and started using Inotify to provide a list of changes made. Got a shell script which is processing the list of changes and sends only them to peers. This script is included in my forked version.
Since then I got time stamps fully fixed and timestamps being always at nanosecond level. Now working on making parallelism.
This will allow sending changes to peers in parallel and make large number of files being sent to peers faster.
Since MySQL and PostgreSQL allow avoiding table lock on deletions/updates - initials tests were done using them. MySQL was slowest even with InnoDB. But PostgreSQL with parallel updates was not going any faster than SQLite single threaded up to 5 peers .
Thus it was chosen to continue using SQLite but with Deletes delayed while changes being sent. This has been successfully tested on a separate branch and will be implemented in my fork Master branch in the coming days.
It will be sort of Hack, but working stable. A proper implementation would require nearly full rewrite of Csync2 code as it is using global variables in many places it should not and code is written not the best way.
Ideally I would like to get a team of decent developers and get Csync2 a full revamp, making it Csync3 where it would be truly multithreaded, cleaned up code, proper connections with up to date coding standards.
Idealy it would listen to Inotify directly and would send transfers as they happen without queueing them up without any delay and SQLite writes would be done after file transfer is finished.