terminal Enable marks for Stable builds

maintainer note: hijacking thread to track bugs we need to fix before enabling marks for Stable builds

Windows Terminal version

1.16.10262.0

Windows build number

10.0.22621.0

Other Software

I am attempting to add a scroll mark, which I believe should appear as a notch on the scroll bar, per https://github.com/microsoft/terminal/pull/12948.

No method to create a scroll mark results in a visible notch in the scroll bar.

I have "experimental.showMarksOnScrollbar": true enabled on my profile

Steps to reproduce

Attempt 1 - echo (what I want so I can make my prompt auto-mark)

The docs have a screenshot of the relevant text rather than characters:

setmark

I had to OCR the above image, and I've checked it a couple of times after @ / 0 being switched, but I'm relatively sure it is:

echo -e 'Mark this Line\x1b]1337;SetMark\x07'

I have "experimental.showMarksOnScrollbar": true enabled on my profile but it doesn't seem to work. I expected a mark on the scroll bar. to appear after running this command.

Attempt 2 - do... something I don't understand

There is also this animated GIF (see docs for animation).

I do not understand how echo "mark 1" is making a mark. The docs mention there is an addMark action but I do not understand how mark 1 and mark 2 etc are related to the addMark action.

Run 'Add Mark' action manually.

I have tried running the AddMark action manually as well:

again I can't get marks to appear. Windows Terminal 1.16.10262.0.

Thanks for understanding - I'm really excited to try this feature, and I appreciate you creating it. ❤️

Expected Behavior

Get a small notch on the scroll bar

Actual Behavior

The scroll bar remains unchanged.

Mar 28 '23 14:03 mikemaccana

Alrighty so I don't know why Attempt 1 is failing for you. It's very possible that "experimental.showMarksOnScrollbar": true might be in the wrong place? I also don't remember if that hot-reloads. I just tried copy-pasting exactly that and got:

and my profiles are set up like:

    "profiles":
    {
        "defaults":
        {
            "experimental.autoMarkPrompts": true,
            "experimental.showMarksOnScrollbar": true,
        },

RE: Attempt 2: That's probably demoing "experimental.autoMarkPrompts": true. That will automatically mark any line where you press enter.

RE: Attempt 3: That really makes me think that experimental.showMarksOnScrollbar is in the wrong place, somehow. Or if you added it to wone particular profile, it might have been on the wrong one. Those Ubuntu profiles can be ~annoying~ tricksy (#12961 et al. )

Mar 28 '23 14:03 zadjii-msft

I'll add:

I'm using

PROMPT $e]133;D$e\$e]133;A$e\$e]9;9;$P$e\$e[107;30m$T]$e[97;46m$G$P$e[36;49m$G$e[0m$_$e[0m$e[94m%username%$e[0m@$e[32m%computername%$e[0m$G$e]133;B$e

as my CMD prompt, and shell-integration-prompt.ps1 for my powershell. Those have a couple more shell integration sequences we lit up in 1.17 that are gonna make this even better in the nearish future. I haven't taken the time to do the same for bash yet, though I mean to soon^TM. As a spoiler: (that's all in 1.17)

Mar 28 '23 14:03 zadjii-msft

Thanks @zadjii-msft. Settings were / still are under the profiles.defaults key, full settings are at end of this comment:

Using your CMD prompt, I still don't get marks:

And here's settings.json:

{
  "$help": "https://aka.ms/terminal-documentation",
  "$schema": "https://aka.ms/terminal-profiles-schema",
  "actions": [
    {
      "command": {
        "action": "scrollToMark",
        "direction": "last"
      },
      "keys": "ctrl+pgdn",
      "name": "Last mark"
    },
    {
      "command": "paste",
      "keys": "ctrl+v"
    },
    {
      "command": {
        "action": "copy",
        "singleLine": false
      },
      "keys": "ctrl+c"
    },
    {
      "command": "find",
      "keys": "ctrl+shift+f"
    },
    {
      "keys": "ctrl+up",
      "command": { "action": "scrollToMark", "direction": "previous" },
      "name": "Previous mark"
    },
    {
      "keys": "ctrl+down",
      "command": { "action": "scrollToMark", "direction": "next" },
      "name": "Next mark"
    },
    {
      "keys": "ctrl+pgup",
      "command": { "action": "scrollToMark", "direction": "first" },
      "name": "First mark"
    },
    {
      "keys": "ctrl+pgdn",
      "command": { "action": "scrollToMark", "direction": "last" },
      "name": "Last mark"
    },
    { "command": { "action": "addMark" } },
    { "command": { "action": "addMark", "color": "#ff00ff" } },
    { "command": { "action": "addMark", "color": "#0000ff" } },
    { "command": { "action": "clearAllMarks" } },
    {
      "command": {
        "action": "newTab",
        "index": 0
      },
      "keys": "ctrl+t"
    },
    {
      "command": "unbound"
    },
    {
      "command": {
        "action": "splitPane",
        "split": "auto",
        "splitMode": "duplicate"
      },
      "keys": "alt+shift+d"
    },
    {
      "command": {
        "action": "scrollToMark",
        "direction": "previous"
      },
      "keys": "ctrl+up",
      "name": "Previous mark"
    },
    {
      "command": {
        "action": "scrollToMark",
        "direction": "next"
      },
      "keys": "ctrl+down",
      "name": "Next mark"
    },
    {
      "command": {
        "action": "scrollToMark",
        "direction": "first"
      },
      "keys": "ctrl+pgup",
      "name": "First mark"
    }
  ],
  "copyFormatting": "none",
  "copyOnSelect": true,
  "defaultProfile": "{2c4de342-38b7-51cf-b940-2309a097f518}",
  "profiles": {
    "defaults": {
      "experimental.autoMarkPrompts": true,
      "experimental.showMarksOnScrollbar": true
    },
    "list": [
      {
        "antialiasingMode": "cleartype",
        "bellStyle": "window",
        "font": {
          "face": "Operator Mono SSm"
        },
        "guid": "{2c4de342-38b7-51cf-b940-2309a097f518}",
        "hidden": false,
        "historySize": 10000,
        "icon": "ms-appdata:///roaming/ubuntu.png",
        "name": "Ubuntu",
        "opacity": 81,
        "source": "Windows.Terminal.Wsl",
        "useAcrylic": true
      },
      {
        "guid": "{574e775e-4f2a-5b96-ac1e-a2962a402336}",
        "hidden": false,
        "name": "PowerShell",
        "source": "Windows.Terminal.PowershellCore"
      },
      {
        "guid": "{61c54bbd-c2c6-5271-96e7-009a87ff44bf}",
        "hidden": false,
        "name": "Windows PowerShell"
      },
      {
        "guid": "{0caa0dad-35be-5f56-a8ff-afceeeaa6101}",
        "hidden": false,
        "name": "Command Prompt"
      },
      {
        "guid": "{b453ae62-4e3d-5e58-b989-0a998ec441b8}",
        "hidden": false,
        "name": "Azure Cloud Shell",
        "source": "Windows.Terminal.Azure"
      }
    ]
  },
  "schemes": [
    {
      "background": "#0C0C0C",
      "black": "#0C0C0C",
      "blue": "#0037DA",
      "brightBlack": "#767676",
      "brightBlue": "#3B78FF",
      "brightCyan": "#61D6D6",
      "brightGreen": "#16C60C",
      "brightPurple": "#B4009E",
      "brightRed": "#E74856",
      "brightWhite": "#F2F2F2",
      "brightYellow": "#F9F1A5",
      "cursorColor": "#FFFFFF",
      "cyan": "#3A96DD",
      "foreground": "#CCCCCC",
      "green": "#13A10E",
      "name": "Campbell",
      "purple": "#881798",
      "red": "#C50F1F",
      "selectionBackground": "#FFFFFF",
      "white": "#CCCCCC",
      "yellow": "#C19C00"
    },
    {
      "background": "#012456",
      "black": "#0C0C0C",
      "blue": "#0037DA",
      "brightBlack": "#767676",
      "brightBlue": "#3B78FF",
      "brightCyan": "#61D6D6",
      "brightGreen": "#16C60C",
      "brightPurple": "#B4009E",
      "brightRed": "#E74856",
      "brightWhite": "#F2F2F2",
      "brightYellow": "#F9F1A5",
      "cursorColor": "#FFFFFF",
      "cyan": "#3A96DD",
      "foreground": "#CCCCCC",
      "green": "#13A10E",
      "name": "Campbell Powershell",
      "purple": "#881798",
      "red": "#C50F1F",
      "selectionBackground": "#FFFFFF",
      "white": "#CCCCCC",
      "yellow": "#C19C00"
    },
    {
      "background": "#000000",
      "black": "#0C0C0C",
      "blue": "#0037DA",
      "brightBlack": "#767676",
      "brightBlue": "#3B78FF",
      "brightCyan": "#61D6D6",
      "brightGreen": "#16C60C",
      "brightPurple": "#B4009E",
      "brightRed": "#E74856",
      "brightWhite": "#F2F2F2",
      "brightYellow": "#F9F1A5",
      "cursorColor": "#FFFFFF",
      "cyan": "#3A96DD",
      "foreground": "#FFFFFF",
      "green": "#13A10E",
      "name": "Color Scheme 10",
      "purple": "#881798",
      "red": "#C50F1F",
      "selectionBackground": "#FFFFFF",
      "white": "#CCCCCC",
      "yellow": "#C19C00"
    },
    {
      "background": "#000000",
      "black": "#0C0C0C",
      "blue": "#0037DA",
      "brightBlack": "#767676",
      "brightBlue": "#3B78FF",
      "brightCyan": "#61D6D6",
      "brightGreen": "#16C60C",
      "brightPurple": "#B4009E",
      "brightRed": "#E74856",
      "brightWhite": "#F2F2F2",
      "brightYellow": "#F9F1A5",
      "cursorColor": "#FFFFFF",
      "cyan": "#3A96DD",
      "foreground": "#FFFFFF",
      "green": "#13A10E",
      "name": "Color Scheme 11",
      "purple": "#881798",
      "red": "#C50F1F",
      "selectionBackground": "#FFFFFF",
      "white": "#CCCCCC",
      "yellow": "#C19C00"
    },
    {
      "background": "#282C34",
      "black": "#282C34",
      "blue": "#61AFEF",
      "brightBlack": "#5A6374",
      "brightBlue": "#61AFEF",
      "brightCyan": "#56B6C2",
      "brightGreen": "#98C379",
      "brightPurple": "#C678DD",
      "brightRed": "#E06C75",
      "brightWhite": "#DCDFE4",
      "brightYellow": "#E5C07B",
      "cursorColor": "#FFFFFF",
      "cyan": "#56B6C2",
      "foreground": "#DCDFE4",
      "green": "#98C379",
      "name": "One Half Dark",
      "purple": "#C678DD",
      "red": "#E06C75",
      "selectionBackground": "#FFFFFF",
      "white": "#DCDFE4",
      "yellow": "#E5C07B"
    },
    {
      "background": "#FAFAFA",
      "black": "#383A42",
      "blue": "#0184BC",
      "brightBlack": "#4F525D",
      "brightBlue": "#61AFEF",
      "brightCyan": "#56B5C1",
      "brightGreen": "#98C379",
      "brightPurple": "#C577DD",
      "brightRed": "#DF6C75",
      "brightWhite": "#FFFFFF",
      "brightYellow": "#E4C07A",
      "cursorColor": "#4F525D",
      "cyan": "#0997B3",
      "foreground": "#383A42",
      "green": "#50A14F",
      "name": "One Half Light",
      "purple": "#A626A4",
      "red": "#E45649",
      "selectionBackground": "#FFFFFF",
      "white": "#FAFAFA",
      "yellow": "#C18301"
    },
    {
      "background": "#002B36",
      "black": "#002B36",
      "blue": "#268BD2",
      "brightBlack": "#073642",
      "brightBlue": "#839496",
      "brightCyan": "#93A1A1",
      "brightGreen": "#586E75",
      "brightPurple": "#6C71C4",
      "brightRed": "#CB4B16",
      "brightWhite": "#FDF6E3",
      "brightYellow": "#657B83",
      "cursorColor": "#FFFFFF",
      "cyan": "#2AA198",
      "foreground": "#839496",
      "green": "#859900",
      "name": "Solarized Dark",
      "purple": "#D33682",
      "red": "#DC322F",
      "selectionBackground": "#FFFFFF",
      "white": "#EEE8D5",
      "yellow": "#B58900"
    },
    {
      "background": "#FDF6E3",
      "black": "#002B36",
      "blue": "#268BD2",
      "brightBlack": "#073642",
      "brightBlue": "#839496",
      "brightCyan": "#93A1A1",
      "brightGreen": "#586E75",
      "brightPurple": "#6C71C4",
      "brightRed": "#CB4B16",
      "brightWhite": "#FDF6E3",
      "brightYellow": "#657B83",
      "cursorColor": "#002B36",
      "cyan": "#2AA198",
      "foreground": "#657B83",
      "green": "#859900",
      "name": "Solarized Light",
      "purple": "#D33682",
      "red": "#DC322F",
      "selectionBackground": "#FFFFFF",
      "white": "#EEE8D5",
      "yellow": "#B58900"
    },
    {
      "background": "#000000",
      "black": "#000000",
      "blue": "#3465A4",
      "brightBlack": "#555753",
      "brightBlue": "#729FCF",
      "brightCyan": "#34E2E2",
      "brightGreen": "#8AE234",
      "brightPurple": "#AD7FA8",
      "brightRed": "#EF2929",
      "brightWhite": "#EEEEEC",
      "brightYellow": "#FCE94F",
      "cursorColor": "#FFFFFF",
      "cyan": "#06989A",
      "foreground": "#D3D7CF",
      "green": "#4E9A06",
      "name": "Tango Dark",
      "purple": "#75507B",
      "red": "#CC0000",
      "selectionBackground": "#FFFFFF",
      "white": "#D3D7CF",
      "yellow": "#C4A000"
    },
    {
      "background": "#FFFFFF",
      "black": "#000000",
      "blue": "#3465A4",
      "brightBlack": "#555753",
      "brightBlue": "#729FCF",
      "brightCyan": "#34E2E2",
      "brightGreen": "#8AE234",
      "brightPurple": "#AD7FA8",
      "brightRed": "#EF2929",
      "brightWhite": "#EEEEEC",
      "brightYellow": "#FCE94F",
      "cursorColor": "#000000",
      "cyan": "#06989A",
      "foreground": "#555753",
      "green": "#4E9A06",
      "name": "Tango Light",
      "purple": "#75507B",
      "red": "#CC0000",
      "selectionBackground": "#FFFFFF",
      "white": "#D3D7CF",
      "yellow": "#C4A000"
    },
    {
      "background": "#000000",
      "black": "#000000",
      "blue": "#000080",
      "brightBlack": "#808080",
      "brightBlue": "#0000FF",
      "brightCyan": "#00FFFF",
      "brightGreen": "#00FF00",
      "brightPurple": "#FF00FF",
      "brightRed": "#FF0000",
      "brightWhite": "#FFFFFF",
      "brightYellow": "#FFFF00",
      "cursorColor": "#FFFFFF",
      "cyan": "#008080",
      "foreground": "#C0C0C0",
      "green": "#008000",
      "name": "Vintage",
      "purple": "#800080",
      "red": "#800000",
      "selectionBackground": "#FFFFFF",
      "white": "#C0C0C0",
      "yellow": "#808000"
    }
  ],
  "themes": []
}

Mar 29 '23 11:03 mikemaccana

Oh my god I totally know what this is.

We never enabled that feature for Stable builds. It's still in Preview-only builds. https://github.com/microsoft/terminal/blob/34aa6aa0d4ffc1dcd3716f6b0dff3b514a4c2d10/src/features.xml#L130-L139

There's still some edge cases that need to be worked out before I'd feel comfortable enabling them for the general public. Would you mind if I repurpose this thread as the "Enable marks for Stable Builds ship list"?

Mar 29 '23 11:03 zadjii-msft

Hah so you thought this was in stable but it wasn't? My thinking was it was announced in 1.15 preview (https://devblogs.microsoft.com/commandline/windows-terminal-preview-1-15-release/#scroll-marks-experimental) so it should definitely be there in 1.16, but I guess that's not how it works. 🤔

And sure repurpose the thread, the cause is "marks aren't enabled for stable ship list" so sure that's a good description.

Mar 29 '23 13:03 mikemaccana

Yea, there's certain features we want to check in to main to let them bake for a while, or let folks try in a preview fashion, but aren't necessarily "stable" yet. Marks is one of those things - you'll see the todo list I edit the OP with. Better to leave these things in Preview-only builds while they bake and we sort out how we exactly want to expose them to our users.

Mar 29 '23 13:03 zadjii-msft

Cool. One suggestion to help people use the new feature is to include the exact terminal sequences we need for pwsh, bash, and cmd in the release notes (so people can copy and paste).

Mar 30 '23 12:03 mikemaccana

Reflow notes:

Add a new enum MarkKind : uint8 { Output=0, Prompt, Command }
- I'm putting Output as 0, because everything is output unless we've been told otherwise
add MarkKind kind to TextAttribute.
add PromptData{ optional<til::color>, optional<int> errorCode } to Row, as an optional

and when we start a prompt, we create prompt data on that row. We're never (in practice) going to have two separate prompts on the same row. We only need one scroll mark per row.

that gives us

easy reflow of "what part of the text buffer is a prompt/command/output",
simple iteration over the marks,
And, we only store the color/error where we need it

Unfiltered Teams notes

[6:30 AM] Mike Griese we just always use the existence of a PromptData on a row to say "there's a prompt that started somewhere on this row" [6:31 AM] Leonard Hecker yup! I think this is the most compact way we can store it while also having the simplicity of the TextAttribute reflow. I'm a tiny bit concerned that this will make reflow of PromptData a little difficult, however. I suppose it would require us to iterate through the copied attributes and check if there was a output-end marker? [6:33 AM] Mike Griese would it though? we're already iterating over old rows once at a time. as we get to each old row, we could just stick the old prompt data onto the equivalent new row [6:34 AM] Mike Griese and like, if there's some contrived buffer that had a long wrapped line with prompts on each, that reflows onto one row, well, we can just use the second row's data [6:34 AM] Leonard Hecker hmm true, but there are potentially multiple new rows if it gets split up. for instance imagine someone uses Win+Right to instantly move a fullscreen window to a half-screen-wide window. this will effectively instantly split up all rows into 2 new rows [6:35 AM] Mike Griese sure, but we still only want one PromptData for the whole prompt [6:35 AM] Mike Griese like, even if you have a multiline prompt, there'd only be one of them, at the very start [6:36 AM] Leonard Hecker hmm yeah that makes sense! honestly, I think once we added both of the above structs, fixing up small issues like this should be simple

addenda:

I'm definitely going to just have the text buffer also be able to hand back a ScrollMark{...} comprised of the same 4 points as before. That way the buffer can store that all as attrs and all that, but then the control layer can just be like "yo give me the relevant regions of text for this prompt"

Mar 22 '24 11:03 zadjii-msft

Won’t that destroy the prompt start and prompt end locations, which we use for some of our detection features? Or would we figure those out after the fact by checking out where Prompt ends and Commans begins?

Mar 22 '24 13:03 DHowett

@DHowett I was thinking that the PromptData on the Row is just like, a bookmark. "On this row, a prompt starts". Then we iterate over the attr runs themselves (possibly wrapping if need be) to gather up the actual text of the prompt.

We honestly could just iterate over the attr runs directly to do this, but the PromptData is there then to stash away the exit code (and color, or the literal command text, or what have you), without us needing to stick it into the TextAttributes themselves.

(we can also extend MarkKind for other kinds of "markup", ala https://gitlab.freedesktop.org/terminal-wg/specifications/-/merge_requests/6#f6de1e5703f5806d0821d92b0274e895c4b6d850)

Mar 22 '24 15:03 zadjii-msft

terminal terminal copied to clipboard

Enable marks for Stable builds

Windows Terminal version

Windows build number

Other Software

Steps to reproduce

Attempt 1 - echo (what I want so I can make my prompt auto-mark)

Attempt 2 - do... something I don't understand

Run 'Add Mark' action manually.

Expected Behavior

Actual Behavior

terminal
terminal copied to clipboard