^M in the process buffer

[yitang]

I'm not sure if this is related to dired-rsync, but I see a lot of ^M character in the process buffer. I couldn't find a solution to get rid of them to have a nice process bar in the buffer.

I wonder if you have any ideas?

[stsquad]

I expect it is a a CR (without a newline). However we do extract the completion percentage in dired-rsync--filter and expose it via dired-rsync-modeline-status for the modeline. You could do something with that?

[linktohack]

Just happen to see this as well, and this patch works for me

  (el-patch-feature dired-rsync)

  (with-eval-after-load 'dired-rsync
    (el-patch-defun dired-rsync--do-run (command details)
      "Run rsync COMMAND in a unique buffer, passing DETAILS to sentinel."
      (let* ((buf (format "%s @ %s" dired-rsync-proc-buffer-prefix (current-time-string)))
	     (el-patch-add (default-process-coding-system '(mac . mac)))
	     (proc (start-process-shell-command "*rsync*" buf command)))
	(set-process-sentinel
	 proc
	 #'(lambda (proc desc)
	     (dired-rsync--sentinel proc desc details)))
	(set-process-filter
	 proc
	 #'(lambda (proc string)
	     (dired-rsync--filter proc string details)))
	(dired-rsync--update-modeline)))

Basically the coding system is wrongly recognised.

[stsquad]

So is this a bug in the underlying process spawning code?

[xor-xor]

I think that this is slightly more nuanced ;) Here's my take: dired-rsync calls start-process-shell-command, which in turn calls start-process, which calls make-process, and then we finally reachfind-operation-coding-system (called as a C function) - if this last call is unable to succeed, then default-process-coding-system is used, which on my mac with Emacs 28.2 happens to be (utf-8-unix . utf-8-unix).

Moreover, if we look at the docstring of find-operation-coding-system function, we will find this:

This function looks up what is specified for TARGET in
`file-coding-system-alist', `process-coding-system-alist',
or `network-coding-system-alist' depending on OPERATION.
They may specify a coding system, a cons of coding systems,
or a function symbol to call.

On my mac, process-coding-system-alist is nil by default, but when I change it to e.g.:

(setq process-coding-system-alist '(("sh" . mac)))

...then I'm getting nicely formatted output in the process buffer :)

So, what I'm trying to say is that there are many ways to skin a cat with Emacs ;) As you can see, the same result can be achieved by manipulating default-process-coding-system (as per @linktohack's answer) or process-coding-system-alist - and who knows what else. Which one should be considered as "canonical" and who is "responsible" for setting those values (i.e. you or your users)? Are there other ways of spawning subprocesses, which allow to explicitly specify the coding system for input/output? These are the questions that I'd like to answer, but I didn't have time to dig further, so I'm leaving my observations here.

[yitang]

@xor-xor thanks for the fix. the progress bar on macOs looks nice. but now i see the same issue on linux. i will follow up after a few experiments.

[yitang]

so on my debian system, it's running gnome. the progress bar got fixed by using the mac's coding system below. see the highlighted line with the changes i made.

Is there a way to fix it for all systems?

note on debian,

1. I have default-process-coding-system set to (utf-8-unix . utf-8-unix)
2. process-coding-system-alist set to nil

Thanks, Yi

[xor-xor]

Is there a way to fix it for all systems?

I guess there is, by e.g. extending this condition that you've highlighted. But first, I'd like to establish that it is indeed broken, as the need to use :coding 'mac on Linux looks a bit suspicious to me. Having said that, I need some time to examine it (say, couple of days), as I don't have a Linux box with Emacs at hand right now.

What is your version of Emacs, BTW?

[yitang]

I'm running Emacs 29.0.60. I was surprised to see :coding 'mac worked and I agree with you.

let me know if you want me to do more tests now or later.

Thanks, Y

[xor-xor]

@yitang after some digging, now I understand what is really happening here.

Those CR characters coming from rsync reporting its progress have nothing to do with macOS or Linux, as they are meant to be interpreted literally (i.e. "return to the beginning of line, so the next progress update will be able to overwrite the last one") - and this is exactly what is happening when you run rsync in "normal" terminal with e.g. --info=progress2.

But to emulate such behavior, we should either modify dired-rsync--filter, or add a dedicated function as another filter. I'm not sure if it's worth the effort, given how short-lived those buffers are.

And that brings us to the :coding 'mac, which only coincidentally provides a quick & dirty workaround, as it leverages the fact that Classic Mac OS systems (i.e. those old, pre-Mac OS X ones) were using CR as a line separator - and this setting just translates CR chars to newlines. As a side effect of sorts, the consecutive progress updates from rsync are written as separate lines (instead of overwriting the predecessor), which is still way better than having them concatenated into a single, long line. So, in short, there's nothing "wrong" in using :coding 'mac in our context on Linux - it only would look weird in the code ;)

Having said that, I'll try to address this issue by using the former approach (i.e. filters) - if it would turn out to be infeasible, I'll prepare a PR extending the workaround based on :coding 'mac.

[stsquad]

I have an idea which might kill two birds with one stone - lets delete everything upto the newline. This will help solve #39

[stsquad]

Can you review the proposed fix please?