`!` is escaped in v5, but wasn’t in v4

[LucasLarson]

Is there an existing issue for this?

• [X] I have searched the existing issues

Update the issue title

• [X] I have updated the title

Expected Behavior

open a terminal window and load this amazing plugin and then enter s then r then n then space:

$ srn

...and have it transformed as it was in v4:

$ sed -e ':a' -e 'N' -e '$! b a' -e 's/\n//g'

Actual Behavior

the text is transformed into the following since v5 (the escaped exclamation point is new)

$ sed -e ':a' -e 'N' -e '$\! b a' -e 's/\n//g'

Steps To Reproduce

1. add an abbreviation with an exclamation in it
2. try to quote it using double quotes, single quotes, and even by way of printf \\41. Here’s some examples I can’t force to work no matter the means I use to try and hide the ! from zsh-abbr and zsh: try filling your $ABBR_USER_ABBREVIATIONS_FILE with the following, all of which worked fine in v4

$ cat $ABBR_USER_ABBREVIATIONS_FILE
abbr one='find . ! -name example'
abbr two='sed -e '\'':a'\'' -e '\''N'\'' -e '\''$! b a'\'' -e '\''s/\n//g'\'''
abbr two='sed -e '\'':a ;          N ;          $! b a ;          s/\n//g'\'''
abbr two='sed -e '\'':a ;          N ;          $! b a ;          s/\n//g'\'''
abbr --global fng2="$(printf -- 'find -- . -mindepth 1 ! -name '\''.git'\'' ! -path '\''*/.git/*'\'' ! -name '\''.DS_Store'\''\n')"
abbr --global fng3='find -- . -mindepth 1 ! -name '\''.git'\'' ! -path '\''*/.git/*'\'' ! -name '\''.DS_Store'\'''
abbr --global fng='find -- . -mindepth 1 ! -name '\''.git'\'' ! -path '\''*/.git/*'\'' ! -name '\''.DS_Store'\'''
abbr --global srn='sed -e '\'':a; N; $! b a; s/\\n//g'\'''

3. when testing any of the abbreviations, notice that the exclamation point retains at least one level of escapedness. This is fine in find where find . \! -path xyz and find . ! -path xyz are interpreted as the same command. It does however break an already-escaped exclamation point
4. I am able to get a partial workaround by adding the following line to my config, but then my problem is like so:

cat $ABBR_USER_ABBREVIATIONS_FILE
abbr --global srn1='sed -e :a -e N -e \$\!b\ a -e s/\\\n//g'

which expands into the following, which includes a literal newline:

$ sed -e :a -e N -e \$\!b\ a -e s/\\
//g

Environment

zsh-abbr version 5.0.0
zsh 5.9 (x86_64-apple-darwin22.1.0)
OSTYPE darwin22.1.0

Using both iTerm (Build 3.5.20230225-nightly) and kitty (Build 3.5.20230225-nightly), getting the same results.

Installation method

Manual

Installation method details

No response

Anything else?

Pretty sure the problem begins in either 30216ba7ba or 2503546976.

Thank you for this solid, serious, well-documented plugin.

[olets]

Thanks for the report and the kind words. Glad the plugin is useful for you!

I can reproduce both the reported v4 behavior and the reported v5 behavior.

And thanks for looking into the history. My guess is it's actually somewhere in

• #50

but it's good to be reminded that there were additional commits in support of multi-word abbreviations after that PR (that work was a while back!). I think it's very likely that you're right and the change in quote levels is the cause.

Work and life are busy right now. I'm not going to commit to a timeline but I do want to see this solved!

In the meantime, downgrading might be a solution for you https://zsh-abbr.olets.dev/migrating-between-versions.html#downgrading-v5-to-v4

--

If you or someone else is interested in helping out, let me know. The quoting is touchy, and a change will impact several commands. Imo it'd be okay if the v5 solution does require writing expansions differently from how they were written in v4.

[olets]

@LucasLarson I recently started a policy of listing as part of the "community" reporters of legitimate bugs (https://zsh-abbr.olets.dev/community/; #83). Would you like to be added?

[LucasLarson]

downgrading might be a solution

I’m determined to stay on the bleeding edge – to remain on version 5 – but I would like to mention that

1. it’s spectacular to see a plugin with usage documentation, and
2. it’s even better when it’s a man page. But to have
3. documentation on how to roll back to a previous version? Wow. 🤩

part of the "community"

Sounds great – thank you!

---

In the meantime, I might get by with something appalling like so:

alias -g srn='
  printf -- '\''sed -e '\''\'\'''\'':a'\''\'\'''\'' -e '\''\'\'''\''N'\''\'\'''\'' -e '\''\'\'''\''\044\041 b a'\''\'\'''\'' -e '\''\'\'''\''s/\\n//g'\''\'\'''\'''\''\\n 2>/dev/null;
  printf -- '\''sed -e '\''\'\'''\'':a'\''\'\'''\'' -e '\''\'\'''\''N'\''\'\'''\'' -e '\''\'\'''\''\044\041 b a'\''\'\'''\'' -e '\''\'\'''\''s/\\n//g'\''\'\'''\'''\'' | pbcopy
'

[olets]

@all-contributors please add @LucasLarson for bug

[allcontributors[bot]]

@olets

I've put up a pull request to add @LucasLarson! :tada:

[olets]

Spent some time on this. No solve yet, and I'm still busy so won't return to it for a while.

Not a safe solution for everyone, but @LucasLarson this might work for you… as long as you rarely need to type a command with \! (by default there's always control space for when you do)

# in zshrc

custom-abbr-expand-and-space() {
  abbr-expand-and-space
  LBUFFER=${LBUFFER:gs/\\!/!} # replace all `\!` with `!`
}
zle -N custom-abbr-expand-and-space
bindkey " " custom-abbr-expand-and-space # must be *after* loading zsh-abbr

--

For future reference: found this, which may point in the right direction:

% abbr exclamation=!
% cat $ABBR_USER_ABBREVIATIONS_FILE  | grep exclamation
abbr "exclamation"="\!"
% abbr | grep exclamation
"exclamation"="\\\!"
% abbr other=other
% cat $ABBR_USER_ABBREVIATIONS_FILE  | grep exclamation
abbr "exclamation="\\!"

[LucasLarson]

This workaround is great – thank you!

Thanks also for adding more breadcrumbs.

[qadzek]

I encountered a similar issue, when trying to create an abbreviation for sudo !! to re-execute the last command with sudo privileges.

abbr 'sbb'='sudo !!^' (mnemonic: sudo bang bang) is expanded to sudo \!\!. Note that I am using the caret as ABBR_EXPANSION_CURSOR_MARKER.

When trying the workaround, the message abbr-expand-and-space is deprecated. is inserted into my terminal every time I hit space.

This abbreviation is similar to sudo !! and avoids using exclamation marks: abbr sbb="sudo \$(fc -ln -1)^"

[olets]

When trying the workaround, the message abbr-expand-and-space is deprecated. is inserted into my terminal every time I hit space.

@qadzek thanks for reporting. To clarify, does deleting abbr 'sbb'='sudo !!^' resolve that problem?

Note that I am using the caret as ABBR_EXPANSION_CURSOR_MARKER.

Always interested to hear the different ways people use abbreviations. What's the context where you want to follow sbb with no space and more text?

[qadzek]

Thanks for your reply.

To clarify, does deleting abbr 'sbb'='sudo !!^' resolve that problem?

It looks like the custom-abbr-expand-and-space snippet is a bit outdated. Replacing abbr-expand-and-space inside the function by abbr-expand-and-insert resolves the problem. Entering sbb<space> now expands as expected.

Besides using custom-abbr-expand-and-space to expand on Space, I created this function so that my abbreviation also expands on Enter:

custom-abbr-expand-and-enter() {
  abbr-expand-and-accept
  LBUFFER=${LBUFFER:gs/\\!/!} # replace all `\!` with `!`
}
zle -N custom-abbr-expand-and-enter
bindkey "^M" custom-abbr-expand-and-enter # must be *after* loading zsh-abbr

[qadzek]

Always interested to hear the different ways people use abbreviations. What's the context where you want to follow sbb with no space and more text?

This is unrelated to this issue and I am unsure if this is expected behavior or a bug. After setting ABBR_SET_EXPANSION_CURSOR=1, are you expected to use the ABBR_EXPANSION_CURSOR_MARKER character (a caret in my case) in all abbreviations? If I don't do so, it seems like all expansions occur twice. E.g. after adding abbr 'foo'='foobar', foo<Space> expands to foobarfoobar.

[olets]

@qadzek

It looks like the custom-abbr-expand-and-space snippet is a bit outdated.

Oh I see yes I forgot about that snippet.

Replacing abbr-expand-and-space inside the function by abbr-expand-and-insert resolves the problem.

That's the right fix. Too late to be useful for you, but I've pushed an update (https://github.com/olets/zsh-abbr/commit/752e9fcc4daff680545c30f8f857913d66f6f5e6) so that the deprecation warning says to use abbr-expand-and-insert.

This is unrelated to this issue and I am unsure if this is expected behavior or a bug. After setting ABBR_SET_EXPANSION_CURSOR=1, are you expected to use the ABBR_EXPANSION_CURSOR_MARKER character (a caret in my case) in all abbreviations?

That's a bug! Thanks for catching it. The immediate workaround is to choose a different cursor marker. zsh treats ^ as a special character in some contexts (! too). Tracking it now in

• #140