Directly piping files into external commands (v2)

[bptato]

Includes the following:

• Stream input into mailcap entries without saving first where possible.
• README.mailcap english translation.

This is a simpler patch than my previous attempt^1. PIPE_LINK and x-nosaveinput have been dropped. Instead, streaming is enabled for the cases where:

• No %s template exists in the mailcap entry, and
• output of external viewers is ignored (or is displayed through needsterminal).

Another improvement over the previous patch: UFhalfclose is used instead of UFclose, so that the parent process does not wait for background external viewer processes to finish downloading.

Stuff not implemented:

• I originally wanted to add streaming for copiousoutput/x-htmloutput; while this would be the "right thing" to do, it seems difficult to implement + pointless since w3m does not support incremental rendering anyway. So I decided to stick with the "download first" approach for these cases.
• I wanted to add an environment variable that is set to the target URL before the entry is executed, but in the end I decided to keep the scope of this PR narrower and did not add it. Maybe in a future PR.

[bptato]

Sorry, I messed up a few things; save2tmp was being called twice sometimes + I didn't account for the HAVE_SETPGRP ifdef. Pushing a fix...

[rkta]

Another improvement over the previous patch: UFhalfclose is used instead of UFclose, so that the parent process does not wait for background external viewer processes to finish downloading.

I don't understand this. AFAIS the only difference between UFclose and UFhalfclose is, that the latter decides based on the scheme which close function to call. With any scheme other than SCM_FTP or SCM_NEWS/_NNTP is will call UFclose.

[bptato]

Rene Kita wrote:

I don't understand this. AFAIS the only difference between UFclose and UFhalfclose is, that the latter decides based on the scheme which close function to call. With any scheme other than SCM_FTP or SCM_NEWS/_NNTP is will call UFclose.

Indeed. My understanding is that for the other stream types, UFclose just closes a FILE. On the other hand, FTP's UFclose waits for the server to respond through closeFTPdata, so that would get the browser stuck until the download is finished.

You can see the same pattern of UFhalfclose used to avoid this problem in doFileSave.

(Admittedly, I haven't tested it with NEWS/NNTP. I had assumed the mechanism is similar, and indeed, there is a news_quit function in news.c that talks to the server. However, the news->rf InputStream doesn't seem to override basic_close, so I don't really see what is going on there.)

[rkta]

On Wed, Jul 31, 2024 at 11:11:45AM -0700, bptato wrote:

Rene Kita wrote:

I don't understand this. AFAIS the only difference between UFclose and UFhalfclose is, that the latter decides based on the scheme which close function to call. With any scheme other than SCM_FTP or SCM_NEWS/_NNTP is will call UFclose.

Indeed. My understanding is that for the other stream types, UFclose just closes a FILE. On the other hand, FTP's UFclose waits for the server to respond through closeFTPdata, so that would get the browser stuck until the download is finished.

You can see the same pattern of UFhalfclose used to avoid this problem in doFileSave.

Interesting, I will need to have a closer look. This has some smell. I'd say we could refactor the code so that your need_halfclose is not needed anymore. This is probably out of scope for this PR, though.

(Admittedly, I haven't tested it with NEWS/NNTP. I had assumed the mechanism is similar, and indeed, there is a news_quit function in news.c that talks to the server. However, the news->rf InputStream doesn't seem to override basic_close, so I don't really see what is going on there.)