Option to print file paths as file URLs

[sanmai-NL]

Printing file paths as file URLs is very handy since you can click on them and the file will open. This happens in Konsole at least.

[BurntSushi]

Do other command line tools have an option like this? I don't think I've seen anything like it.

[sanmai-NL]

@BurntSushi: I don't know off the top of my head. You also encounter such such links while paging log output, for example.

To be clear, this isn't a question but a feature request. 🙂

Similar feature requests

• robotframework/robotframework#2435
• ...

Added value

• Quicker access to files due to clickable paths in KDE Konsole, macOS Terminal, ...
• URLs can be read by wget and curl

[kpp]

echo -e "/path1\n/path2" | xargs -n1 -I{} echo file://{}
file:///path1
file:///path2

You may do the same for rg without modifying it.

[BurntSushi]

I think I'm going to close this. I don't see a lot of other tools doing this, and it should be possible to write a wrapper script to do this for you if you really want it.

[sanmai-NL]

@BurntSushi: Important interesting precedent: https://github.com/systemd/systemd/blob/903511951892008b991991962fef2e8ecc65bc6c/NEWS#L220-L232.

[adrianocr]

I think this should be reopened and properly considered. Its insanely useful. Certain terminal apps such as iterm2 does it by default. But it could work across ALL of them if rg outputted links with the file:// syntax.

[alex]

@BurntSushi I am not quite sure how this is implemented, but I just learned that ls has a feature ls --hyperlink=auto (or always or never), which causes it to emit things with ANSI shell magic that makes them clickable links in my terminal. Here's an example:

alexgaynor@penguin ~/p/step-expirements> ls --hyperlink=always example.py
example.py
alexgaynor@penguin ~/p/step-expirements> ls --hyperlink=always example.py | xxd
00000000: 1b5d 383b 3b66 696c 653a 2f2f 7065 6e67  .]8;;file://peng
00000010: 7569 6e2f 686f 6d65 2f61 6c65 7867 6179  uin/home/alexgay
00000020: 6e6f 722f 7072 6f6a 6563 7473 2f73 7465  nor/projects/ste
00000030: 702d 6578 7069 7265 6d65 6e74 732f 6578  p-expirements/ex
00000040: 616d 706c 652e 7079 0765 7861 6d70 6c65  ample.py.example
00000050: 2e70 791b 5d38 3b3b 070a                 .py.]8;;..

If ls can have nice things, surely a program as awesome as rg can 😂

[saleemrashid]

This is fairly comprehensive documentation on the hyperlink feature that @alex mentioned: https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda

[mx-psi]

Any chance this will be reopened? As mentioned above, there is widespread support for hyperlinks on modern terminal emulators and tools like ls have built-in support for this.

[kovidgoyal]

FYI: Here is a wrapper implementation that actually allows you to click on results and open them in your editor at the matched line. https://sw.kovidgoyal.net/kitty/kittens/hyperlinked_grep.html

Note that it works with the kitty terminal.

[adrian-gierakowski]

@BurntSushi would you be so kind and reconsider? it would be amazing to have it supported out of the box and not have to use wrappers

[bluss]

The following things seem to be needed for supporting hyperlinked file paths in ripgrep. I'm just writing for no reason, maybe it's useful.

• Resolve paths to absolute paths (I would suggest do so without touching the filesystem, don't want canonicalization) and url escape paths to be printable-ascii-safe as file:// URIs.
• Lookup current hostname (bonus feature but should preferably be in the URL according to the spec, this seems to be a bit of a layer-busting complication)
• Add cli parameter --hyperlink={always|ansi|auto|never} (I don't know if ansi is useful here, but I think so)
• Use the hyperlink escapes in the printer crate. I'm not sure if it's a great fit to try to put it inside termcolor - even if the spec says it works very similarly to colors.

This feature is supported with the same escape code in Linux (various terminals including gnome-terminal), Windows (microsoft/terminal), macos (iTerm2). The pager less supports it in new versions.

[BurntSushi]

@bluss Thanks for laying that out! Some quick feedback:

• If possible, I'd like to avoid the hostname lookup aspect of this. You're spot on about it being layer busting. I think the key thing here is, what does the hostname buy us? i.e., What do we lose if we don't bother with it.
• Putting this inside of termcolor might be nice to be honest, especially if it's an ANSI escape sequence. Or at least, putting the emission of that ANSI escape inside termcolor would be good. Having it be outside of termcolor would probably be more problematic.
• When supporting the legacy Windows console, I think we would just not emit the escape sequence, like we do now for things not supported by the Windows console.

I think that's all I can think of for right now. I'll re-open.

[NightMachinery]

@burntsushi The hostname can enable usage on SSH. With the hostnames present, one can configure their editor to open the remote files (both Vim and Emacs support this very well, and VSCode kind of supports it as well). If there is a performance concern, just cache the hostname every, say, week.

[BurntSushi]

The concern isn't performance. Folks asking for file URIs can afford to pay for one hostname query per ripgrep execution. The issue (in my head) is abstraction. It likely means refactoring. I don't know how severe it is.

[bluss]

Thanks! The main feature of the hostname is just for the terminal to know if it's a local path or not, as I understand it, so that it can avoid trying to open a remote path locally. (I haven't seen other features that are worth the bother, but they might come, too.) Just for fun, here's a snippet for making hyperlinks in the shell printf '\e]8;;%s\e\\%s\e]8;;\e\\\n' "$LINK" "$TITLE"

[kovidgoyal]

If you leave out hostname, you make it impossible for things like this to function: https://sw.kovidgoyal.net/kitty/kittens/remote_file.html

That allows the user to click on hyperlinks pointing to a file on a remote machine via SSH and have the file edited locally and automatically re-uploaded, seamlessly by just clicking on it.

Please do not leave it out.

[XhstormR]

This feature is really helpful.

[ahmadsamir]

Hello. This would be a good addition to rg. About the hostname part of this, I've never seen a file:/// url with the hostname part specified (I've been using Linux since 2007); yes the spec talks about it, but in the real world it doesn't seem to be used for file:/// urls.

ls has this feature for local files only, right?

I think adding this for local files only would cover 90% of the use-cases.

My 2p's.

[Jackenmen]

ls has this feature for local files only, right?

No, it properly includes the hostname. As said above, it actually allows for existence of useful features such as clicking on the hyperlink in SSH session and opening or saving it with the local editor:

About the hostname part of this, I've never seen a file:/// url with the hostname part specified (I've using Linux since 2007); yes the spec talks about it, but in the real world it doesn't seem to be used for file:/// urls.

Escape sequences for hyperlinks did not exist in 2007. You might have not seen file:// with the hostname part specified in the typical contexts where you've seen file:// (browsers, desktop environments, etc.) but you need to look at this within the context of terminal emulators and within that context, specifying hostname is well-established by this extension of the VT protocol: https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda#file-uris-and-the-hostname

[ahmadsamir]

That would require support in the terminal emulators, to fetch the file using scp, edit it, then reupload it... I don't know what terminal emulators, other than kitty, will actually support that.

My main point is, this can be implemented for local files only now (looking at the above comments the hostname querying seems to be a big obstacle for implementing this), and extended later.

Also note that, even gnome-terminal itself, will just tell you it can't open such kind of url:

Could not open the address "file://foo/bar/baz"
"file" scheme with remote hostname not supported

[Jackenmen]

My main point is, this can be implemented for local files only now

If you want to follow the specification that I linked (and the spec that all the terminals seem to be using), I'm not sure you can as it seems to be quite clear on this:

Web browsers, desktop environments etc. tend to ignore the hostname component of a file://hostname/path/to/file.txt URI. In terminal emulators, such ignorance would lead to faulty targets if you ssh to a remote computer. As such, we don't allow this sloppiness. Utilities that print hyperlinks are requested to fill out the hostname, and terminal emulators are requested to match it against the local hostname and refuse to open the file if the hostname doesn't match (or offer other possibilities, e.g. to download with scp as iTerm2 does).

[ltrzesniewski]

Note that currently, Windows Terminal does not support the hostname:

https://github.com/microsoft/terminal/blob/54dc2c4432857919a6a87682a09bca06608155ed/src/cascadia/TerminalApp/TerminalPage.cpp#L2304-L2309

Which basically means the hostname should be omitted, at least on Windows. Note that the spec requests for utilities to add the hostname, it does not require it.

(I'd like to work on this feature BTW)

[kovidgoyal]

On Sat, Oct 01, 2022 at 11:52:09AM -0700, Lucas Trzesniewski wrote:

Note that currently, Windows Terminal does not support the hostname:

https://github.com/microsoft/terminal/blob/54dc2c4432857919a6a87682a09bca06608155ed/src/cascadia/TerminalApp/TerminalPage.cpp#L2304-L2309

Which basically means the hostname should be omitted, at least on Windows. Note that the spec requests for utilities to add the hostname, it does not require it.

One terminal not supporting it on Windows doesnt mean it should be omitted on windows.

And as has been noted above multiple times, not adding hostname will break functionality in advanced terminal emulators. Both iTerm2 and kitty use hostname to allow hyperlinks to work over ssh.

If ls can output hostnames in its hyperlinks with no ill effects so can rg.

[ltrzesniewski]

One terminal not supporting it on Windows doesnt mean it should be omitted on windows.

Well, the problem on Windows is not really Windows Terminal itself (though that's the main terminal app). The problem is that the shell doesn't support the hostname in file hyperlinks. Any terminal app trying to open a link using the shell will have the same issue. You cannot make hyperlinks broken by default on Windows.

I just tried opening such an URI with the shell, and I got the following error: "The system cannot find the file specified.". Windows Terminal prevents it and displays a more user-friendly error instead.

Here are the options I can see:

• Omit the hostname everywhere (breaks SSH)
• Include the hostname everywhere (breaks Windows Terminal and other apps which use the shell)
• Omit the hostname on Windows, include it everywhere else (good compromise IMO, SSH would work with kitty)
• Omit the hostname if the WT_SESSION environment variable is set (special-cases Windows Terminal)
• Add a command line argument for this (maybe overkill)

[ssbarnea]

There is no need for hostname for local file URI, they only need file:// prefix, followed by the FULL-PATH (whatever is that on windows, especially as running under WSL would mean a unix path). URI cannot have relative paths, not supported by default. Feel free to consult the specification or details at https://datatracker.ietf.org/doc/html/rfc8089#appendix-D.2

[ltrzesniewski]

There is no need for hostname for local file URI

The concern here is running ripgrep through SSH. That would output "local" files paths from the point of view of the remote machine. The terminal would need the hostname to detect those aren't really local.

whatever is that on windows, especially as running under WSL would mean a unix path

Those work:

• file:///C:/SomeDir/SomeFile.txt
• file://localhost/C:/SomeDir/SomeFile.txt
• file://wsl$/Ubuntu/SomeDir/SomeFile.txt

Those don't:

• file://local_hostname/C:/SomeDir/SomeFile.txt

I just realized WSL links work fine with the shell, but Windows Terminal blocks them. I'll open an issue for that.

Ew, actually, remote hostnames work fine, but the local hostname is rejected unless it's literally localhost. 😭

[kovidgoyal]

On Sun, Oct 02, 2022 at 03:10:22AM -0700, Lucas Trzesniewski wrote:

One terminal not supporting it on Windows doesnt mean it should be omitted on windows.

Well, the problem on Windows is not really Windows Terminal itself (though that's the main terminal app). The problem is that the shell doesn't support the hostname in file hyperlinks. Any terminal app trying to open a link using the shell will have the same issue. You cannot make hyperlinks broken by default on Windows.

I dont follow, any halfway decent terminal running on windows should just parse the URI, if it detects that it has a hostname, match it against the hostname of the local machine, if it is local it can remove the hostname from the URI before passing it on to the shell. Or just extract the local file path and pass it to the shell.

There is absolutely zero need for individual terminal applications to try to specialize their output based on intended destination OS, which is anyway not possible to determine, given that they could be running over SSH.

[ltrzesniewski]

@kovidgoyal it seems you're reading this discussion through your mails, so you probably didn't see it, but I have made lots of edits to my posts, and opened an issue with Windows Terminal which requests them to do exactly what you suggest. 🙂

But today, it's unfortunately broken, and we need to deal with it. If you run ripgrep over SSH, the remote process won't be on Windows anyway so it'll output the hostname just fine.

Note that ripgrep already specializes on the OS for much more trivial things:

https://github.com/BurntSushi/ripgrep/blob/4386b8e805e273a9795ad4e256c455c74407c949/crates/printer/src/color.rs#L16-L19

[ltrzesniewski]

I started implementing this in #2322, and have a preview version available if you're interested.

If you'd like to try it out, I made a GitHub workflow which builds ripgrep with this feature. Just open the latest build and download the artifact for your platform (at the bottom of the build summary - unfortunately, many warnings are produced by the rustup action).