hstr stops working on linux >=6.2.0 (depending on kernel config)

[leapfog]

Just wanted to leave a note, that hstr stops working with linux kernel 6.2.0, when CONFIG_LEGACY_TIOCSTI isn't set. So maybe you want to add that information to a trouble shooting guide or FAQ or something.

(re: "stops working": the history can still be browsed, but cannot be inserted anymore)

[BlueMax]

Edit: Never mind, probably only switchable with CONFIG_LEGACY_TIOCSTI=y.

~~How is this meant to be done?~~

This functionality can be changed at runtime with the dev.tty.legacy_tiocsti sysctl. This configuration option sets the default value of the sysctl. https://cateee.net/lkddb/web-lkddb/LEGACY_TIOCSTI.html

It can't be set though.

# sysctl -w dev.tty.legacy_tiocsti=1
sysctl: setting key "dev.tty.legacy_tiocsti": Invalid argument
# echo 1 > /proc/sys/dev/tty/legacy_tiocsti
bash: echo: write error: Invalid argument

Arch Linux 6.2.1

[sxe]

I came here as well cause hstr stopped working. Could we get an official reply on how this situation will be handled? Will hstr be adjusted to work without it? I would rather not enable a legacy option.

Cheers

[rajakesar]

❯ uname -r 6.2.1-arch1-1

❯ date Thu Mar 2 10:38:30 AM EST 2023

Hstr is confirmed NOT working on the above kernel as of the time above.

[jakedane]

If you run zgrep CONFIG_LEGACY_TIOCSTI /proc/config.gz you can see CONFIG_LEGACY_TIOCSTI is not set for 6.2.1-arch1-1. If indeed hstr insertion is not working because CONFIG_LEGACY_TIOCSTI is not set then this is why for 6.2.1-arch1-1.

[mmeier86]

I can confirm that it's the CONFIG_LEGACY_TIOCSTI option. I had the same problem after I updated my Gentoo to 6.2, and after reenabling that option in my Kernel, it started to work again with 6.2.

[leapfog]

Seems fzf's history search is still working, even without CONFIG_LEGACY_TIOCSTI, so there is a way.

[jakedane]

Seems fzf's history search is still working, even without CONFIG_LEGACY_TIOCSTI, so there is a way.

fzf has a bash function around it for the history function, that sets the readline input buffer with the history line that was selected in fzf. The same can be done with hstr!

What I did:

1. Build hstr with DEBUG_NO_TIOCSTI set. You just need to uncomment this line and recompile hstr: https://github.com/dvorka/hstr/blob/master/src/hstr_utils.c#L29. With DEBUG_NO_TIOCSTI set hstr doesn't use the insecure TIOCSTI and instead prints the selected history line to stderr. On Arch Linux I did this with:

auracle download hstr
cd hstr
makepkg -so
sed -i -r 's|//(#define DEBUG_NO_TIOCSTI)|\1|' src/hstr-*/src/hstr_utils.c
makepkg -e
sudo pacman -U hstr-*.zst

2. We don't want the selected history line on stderr, we want it in the readline input buffer. I couldn't fully follow how fzf was doing that but on the wiki there is an example for bash: https://github.com/junegunn/fzf/wiki/Examples#with-write-to-terminal-capabilities. I used the 3rd example, the one starting with # re-wrote the script above, and adjusted it a bit for hstr. Basically: call hstr, redirect its stderr to a file so that can be read back into a variable and then pass that variable to the function from the wiki that sets the readline input buffer. Add this to your ~/.bashrc:

bind '"\C-r": "\C-x1\e^\er"'
bind -x '"\C-x1": __hstr';

__hstr ()
{
	hstr 2> ~/.hstr.tmp
	hstr_tmp=$(< ~/.hstr.tmp)
	__ehc "$hstr_tmp"
}

__ehc()
{
if
        [[ -n $1 ]]
then
        bind '"\er": redraw-current-line'
        bind '"\e^": magic-space'
        READLINE_LINE=${READLINE_LINE:+${READLINE_LINE:0:READLINE_POINT}}${1}${READLINE_LINE:+${READLINE_LINE:READLINE_POINT}}
        READLINE_POINT=$(( READLINE_POINT + ${#1} ))
else
        bind '"\er":'
        bind '"\e^":'
fi
}

3. And you need to disable the default hstr keybindings in your ~/.bashrc so comment out these lines:

# if this is interactive shell, then bind hstr to Ctrl-r (for Vi mode check doc)
if [[ $- =~ .*i.* ]]; then bind '"\C-r": "\C-a hstr -- \C-j"'; fi
# if this is interactive shell, then bind 'kill last command' to Ctrl-x k
if [[ $- =~ .*i.* ]]; then bind '"\C-xk": "\C-a hstr -k \C-j"'; fi

Now if I open a new terminal, press Ctrl+R, hstr works again and the selected history line is placed on the readline input.

I don't fully understand all of the above — and it can probably be cleaned up and made nicer — but at least I have a working hstr again.

[BlueMax]

CopyQ is another candidate that still works. I'm not sure but i think they load the clipboard and send a keystroke via X11 to paste the text.

[papavlos]

Thanks @jakedane for the workaround. It works! There is a small issue - pressing Enter on selected command in history places the command into input buffer but without automatic execution. I have to press Enter again, but this is very small issue.

[Mte90]

I am on debian sid and stopped working too with 6.2.2-2-siduction-amd64 #1 SMP PREEMPT_DYNAMIC siduction 6.2-2.1 (2023-03-06) x86_64 GNU/Linux

apt source hstr
sed -i -r 's|//(#define DEBUG_NO_TIOCSTI)|\1|' src/hstr_utils.c
debuild -uc -us

Compiled with the patch, it still not work. I just get the command in the terminal after the selection compared to before.

[jakedane]

Compiled with the patch, it still not work. I just get the command in the terminal after the selection compared to before.

That sounds like you did step 1 from https://github.com/dvorka/hstr/issues/478#issuecomment-1454727507 but didn't proceed with step 2 and 3 to modify your .bashrc file (and open a new terminal after).

[Mte90]

So I tried that bash changes and in this way works only Ctrl+r not hh as example (I tried changing the bash-completion script):

#
# Copyright (C) 2014-2022  Martin Dvorak <[email protected]>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Bash completion support for HSTR

# Source this file or install it to /usr/share/bash-completion/completions or /etc/bash_completion.d/
# See https://iridakos.com/tutorials/2018/03/01/bash-programmable-completion-tutorial.html
# help complete
# complete -W "--favorites --kill-last-command --non-interactive --show-configuration --show-zsd-configuration --show-blacklist --version --help" hstr

bind '"\C-r": "\C-x1\e^\er"'
bind -x '"\C-x1": _hstr';

#_hstr()
#{
#        local cur prev OPTS
#        COMPREPLY=()
#        cur="${COMP_WORDS[COMP_CWORD]}"
#        prev="${COMP_WORDS[COMP_CWORD-1]}"
#        case $prev in
#                '-h'|'--help'|'-v'|'--version')
#                        return 0
#                        ;;
#        esac
#        case $cur in
#                -*)
#                        OPTS="--favorites --kill-last-command --non-interactive --show-configuration --show-zsd-configuration --show-blacklist --version --help"
#                        COMPREPLY=( $(compgen -W "${OPTS[*]}" -- $cur) )
#                        return 0
#                        ;;
#        esac
#        compopt -o bashdefault
#        COMPREPLY=( $(compgen -c -- $cur) )
#        return 0
#}

_hstr ()
{
	hstr 2> /tmp/.hstr.tmp
	hstr_tmp=$(< /tmp/.hstr.tmp)
	__ehc "$hstr_tmp"
}

__ehc()
{
if
        [[ -n $1 ]]
then
        bind '"\er": redraw-current-line'
        bind '"\e^": magic-space'
        READLINE_LINE=${READLINE_LINE:+${READLINE_LINE:0:READLINE_POINT}}${1}${READLINE_LINE:+${READLINE_LINE:READLINE_POINT}}
        READLINE_POINT=$(( READLINE_POINT + ${#1} ))
else
        bind '"\er":'
        bind '"\e^":'
fi
}
complete -F _hstr hstr
complete -F _hstr hh

# eof

[leapfog]

While I still hope for hstr to be modified to work without CONFIG_LEGACY_TIOCSTI, kernel 6.2.x has been fixed, so re-enabling it at runtime is now possible:

# sysctl -w dev.tty.legacy_tiocsti=1 # fix hstr
dev.tty.legacy_tiocsti = 1

I only confirmed that with linux-6.2.6 today, but it might also have worked earlier.

[karlovskiy]

I can confirm that with 6.2.5-arch1-1 re-enabling sysctl -w dev.tty.legacy_tiocsti=1 is now also works.

[Mte90]

I can confirm too on Debian sid (siduction) with 6.2.6-1 the hstr package from debian repository works again.

[chrischmo]

Re-enabling sysctl -w dev.tty.legacy_tiocsti=1 also works on OpenSUSE Tumbleweed (6.2.4-1-default) with hstr from the lemmy04 repo.

[dvorka]

@Mte90 @chrischmo @karlovskiy @leapfog @jakedane @papavlos @mmeier86 thank you all for detailed descriptions of the problem, root cause identification and proposed solutions!

I worked on a version of HSTR for Cygwin and WSL where TIOCSTI is not available in the past. If you are able to build HSTR and you use Bash, can you please try #481?

The PR just enables existing HSTR functionality to work w/o TIOCSTI w/ the new define. I did not used it in the past because it requires shell command (and many users will not (be able to) do such configuration).

Anyway if the PR will work for you, I will fix it also for zsh (which I use on 90% of my machines) and release a new version.

Do you think that it would make sense to release HSTR with a TIOCSTI parameter allowing to run version with or without TIOCSTI use? Is there any way how to safely detect TIOCSTI availability? Will it compile on systems with new kernel?

Thank you all your interest in HSTR and your help!

[jakedane]

@dvorka I built hstr with #481 and with LINUX_KERNEL_6 defined, I put the new config in my .bashrc and with that Ctrl+R works properly in Bash. Thank you!

I'm on Arch Linux with kernel 6.2.6. No issue compiling.

I don't know how to safely detect TIOCSTI is available. Probably too hacky and not portable but on Linux if sysctl -ne dev.tty.legacy_tiocsti prints 0 (zero) that means TIOCSTI is disabled.

[dvorka]

Preliminary TIOCSTI design:

• .bashrc/.zshrc function:
  • detects TIOCSTI and based on that:
    • call the right Bash/zsh function which binds to the shortcut
    • hstr command w/ or w/o --no-tiocsti parameter
• hstr binary:
  • detects TIOCSTI and based on that it either uses it or not
    • hstr binary supports both modes
    • #define is not used to compile only one option

---

.bashrc

• detect whether TIOCSTI is supported by the kernel @ Bash
• based on whether TIOCSTI is available bind keyword shortcuts either to hstr command or hstrcygwin Bash function

# detect TIOCSTI
function is_tiocsti {
    if test -w /dev/tty && { stty -echo; echo -n a | tioctl TIOCSTI; stty echo; } >/dev/null 2>&1; then
        echo "TIOCSTI is supported by the kernel."
    else
        echo "TIOCSTI is not supported by the kernel."
    fi
}

# command binding
if is_tiocsti
then
    # if this is interactive shell, then bind hstr to Ctrl-r (for Vi mode check doc)
    if [[ $- =~ .*i.* ]]; then bind '"\C-r": "\C-a hstr -- \C-j"'; fi
    # if this is interactive shell, then bind 'kill last command' to Ctrl-x k
    if [[ $- =~ .*i.* ]]; then bind '"\C-xk": "\C-a hstr -k \C-j"'; fi
else
    if [[ $- =~ .*i.* ]]; then bind -x '"\C-r": "hstrcygwin"'; fi
fi 

---

Detect TIOCSTI from C:

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <termios.h>
#include <sys/ioctl.h>

int main() {
   int fd;
   struct termios t;
   fd = open("/dev/tty", O_RDWR);
   if (fd < 0) {
      perror("open /dev/tty");
      exit(1);
   }
   if (tcgetattr(fd, &t) < 0) {
      perror("tcgetattr");
      exit(1);
   }
   if (!ioctl(fd, TIOCSTI, "a")) {
      printf("TIOCSTI supported\n");
   } else {
      printf("TIOCSTI not supported\n");
   }
   close(fd);
   return 0;
}

[jakedane]

The C code detects TIOCSTI correctly. I tested it both with dev.tty.legacy_tiocsti=0 and dev.tty.legacy_tiocsti=1.

The bash code does not detect it correctly. is_tiocsti always responds "TIOCSTI is supported by the kernel."

Edit: ah, removing the redirect to /dev/null I get tioctl: command not found.

[dvorka]

@jakedane thank you for testing the code! It really helped. As tioctl command does not have to be present on the system, I will always hstr - I will create a hstr parameter which will make the test (and use it in .bashrc/.zshrc).

[dvorka]

FYI working on the fix @ dev-2.7.0 branch.

[hartwork]

Hi!

I'd like to add some information regarding TIOCSTI ioctl that I didn't see mentioned above (from a quick look):

• Allowing use of TIOCSTI has security consequences, e.g. see ttyjack to see trivial privilege escalation through command injection using TIOCSTI in action.
• TIOCSTI is enabled by default in upstream Linux but disabled by default in Arch Linux.
• The sysctl variable dev.tty.legacy_tiocsti could not be set successfully in the past on some systems, including Arch Linux. There was a kernel module built in the mean time to bring it back at https://github.com/kauruus/legacy_tiocsti to workaround the issue but by now it is no longer needed and sysctl should work fine.

Best, Sebastian

[dvorka]

PROGRESS UPDATE: I apologize that the fix is not out yet (busy days). It was a bit painful process, but I finally have the solution for both Bash and Zsh which enables HSTR to work w/o TIOCSTI :relieved:

Zsh:

hstr_notiocsti() {
    zle -I
    { HSTR_OUT="$( { </dev/tty hstr ${BUFFER}; } 2>&1 1>&3 3>&- )"; } 3>&1;
    BUFFER="${HSTR_OUT}"
    CURSOR=${#BUFFER}
    zle redisplay
}
zle -N hstr_notiocsti
bindkey '\C-r' hstr_notiocsti

Bash:

function hstrnotiocsti {
    { HSTR_OUT="$( { </dev/tty hstr ${READLINE_LINE}; } 2>&1 1>&3 3>&- )"; } 3>&1;
    READLINE_LINE="${HSTR_OUT}"
    READLINE_POINT=${#READLINE_LINE}
}
if [[ $- =~ .*i.* ]]; then bind -x '"\C-r": "hstrnotiocsti"'; fi

Having the solution, I just need to polish it a bit and do the release. Stay tuned please.

[krossekrabbe]

Posted hstr_notiocsti() function does not work for me, still does nothing after selecting the history entry. Don't know if I am doing something wrong. I already verified that it is actually being executed on CTRL+R.

fedora 37, kernel 6.2.8-200.fc37.x86_64

sysctl -w dev.tty.legacy_tiocsti=1 is making it work.

[dvorka]

@krossekrabbe apologies for confusion - I shared the function, however, it will work with new version of HSTR. Anyway thank you for your interest in this issue!

[dvorka]

Fixed by aa14f032d4f39b38d7c7ccc042f18400760a3bd7 and released by https://github.com/dvorka/hstr/commit/f370a80e3d76cd74d0f3064e32ef86b0339a402f. Please report any problems!

[EmJotGeh]

The fix only seems to work to some extent. After entering the hstr_notiocsti() function in my .bashrc, the history can be opened via CTRL+R but the selected command now appears in the console and is not executed until ENTER is pressed. This corresponds more to the function of the right cursor key (->) - to display the command and to be able to change it if necessary.

[cmonty14]

Imo the fix is not working as expected. This means I can display bash history and select any line item. However when I hit ENTER the line item will only be copied to console, but not executed.

I'm running Arch Linux with hstr version "3.1.0" (2023-04-18T08:50:00) and Linux Kernel 6.2.12-arch1-1. TIOCSTI is disabled (dev.tty.legacy_tiocsti = 0).

My understanding was that hstr should work /w and w/o TIOCSTI.

[krossekrabbe]

Yes can confirm on fedora, ENTER writes to console but not executes.

But I can live with that for now, close enough :joy: Thanks for the fix @dvorka