terminal

terminal copied to clipboard

Lets talk about restoring the state of the terminal here.

This could arise in two scenarios I'm thinking of:

1. Restoring the state of a previously closed terminal session (i.e. the app was closed due to a restart, and you launch it again)
2. Restoring from a previously closed tab in this session (i.e. you've closed a tab, but not the app.)

1 is certainly harder than 2.

Restoring the buffer contents wouldn't be impossible, but restoring the actual state of the running executable might be impossible. IIRC @malxau has a long email from his work on yori that is relevant to this discussion.

We could certainly restore the buffer contents from file, then display some sort of message saying [Restored from previous session]. That's not that hard.

I think there are possible security considerations with this. With that in mind, making restoring previous output seems an opt-in rather than opt-out feature seems prudent. On Windows of course if the user's profile is BitLocker-protected, maybe it's less of an issue, but regardless worth considering.

I did enjoy this feature of Terminal.app for the few years I used a mac some time ago.

Firstly, welcome to github and the terminal project, @BEAClerbois!

Just adding to the record some of the information @zadjii-msft is referring to, I tried to implement save/restore of state as part of Yori, which is a CMD-like shell. This was opt-in due to the security/privacy issue that @binarycrusader mentioned, and is enabled with an environment variable. See http://www.malsmith.net/yori/guide/#env_yoriautorestart .

In that context, save/restore meant saving both terminal state as well as shell state. Terminal state includes things like the size and position of the window, its contents, colors, and fonts.

Shell state includes things like current directory, environment, and in this case, aliases and command history.

But the real challenge is how to reason about multi-process relationships. One example was given previously: suppose the user launches a CMD window, and within that launches Powershell. Even if both CMD and Powershell were capable of saving their state, the restoration process needs to include a) terminal state; b) a CMD process with its state; c) a Powershell process with its state; d) somehow get the CMD process to wait for the Powershell process to terminate. (d) is very unnatural because on the running system the parent can only reason about the child via a handle or PID, and here neither are known until the child is running. It's possible that I'm not thinking creatively enough, but this sounds a bit like a cyclic dependency, because on the one hand the parent process should be fully restored before the child can accept user input and terminate itself, but on the other hand, the parent process can't really know what to wait for until the child is present.

A clearer example of the same effect would be opening CMD and within that executing "vim foo". The terminal will know about the existence of the vim process, but the meaning of "foo" depends on the current directory that vim was launched from, and that information has to come either from vim or from the parent process, and CMD needs to wait for vim. Recovering things accurately would require coordination from three different programs.

When the multi-process relationship is outside the console, things get messier still. The specific case that affected Yori users which I couldn't see how to handle is the Virtual Filesystem for Git agent, because launching a shell with a correct current directory does not mean the user can interact with the files there unless the agent is running. But the agent can be launched via any mechanism which may be completely unrelated to the console, so there's no knowledge indicating it must be restored. In the context of restoring a recently closed tab this may not be so problematic.

Obviously this is in addition to restoring state which depends on OS configuration which may not be present - having network shares, authentication, a particular IP address, a USB device attached - things that might change between attempting a save and a restore.

Thanks @malxau for the great explanation. As we can see, restoring shell state is a neigh-impossible thing to do totally correct (and why I wanted this to have it's own thread separate from #766).

However buffer content is not impossible to restore. Lets make it opt-in, in addition to opting-in to general window state restore being tracked in #766.

After sleeping on it, I started remembering even nastier cases about process relationships. Consider pipes: "type foo | more" means saving the state of the "type" command and the "more" command and somehow recreating the pipe. This looks like another cyclic dependency: pipes are normally created by the parent, which carefully hands one end to the child and closes it; but here we need some kind of resumable identifier that describes a pipe between two processes and can recreate that. The "type" command, in turn, needs to know where it was within its source stream to resume from; in this example with a source file that's at least possible.

When the source stream is dynamic, finding that point becomes somewhat meaningless. Suppose the command was "tasklist | more" - where should tasklist resume from? Its source stream has completely changed.

Agree with "neigh-impossible", and this feels like something where if we tried to solve it, it would end up being Windows-specific and a large part of the terminal effort is better interoperability with other platforms. It seems to me like if there was a push to solve this on other platforms we can contribute to the conversation, but going it alone doesn't seem like the right thing to do. Users would be left with an experience that only works for some programs, some of the time.

@malxau While all the complications make saving state difficult, I don't feel simply saving the tabs, tab names, panes, and their respective directories should be relatively simple.

Having different session setups could also be possible as an argument to wt, for example I could type win+r wt session foo to open the foo session. Saving could be accomplished via wt session -s foo.

Thinking about the above suggestion, it may be easier to add a section to the options JSON that let's users describe a session layout.

@JamieGhassibi How would the terminal obtain the directory that is active within the shell processes?

I can see how this would work if the session is defined in config, but to capture the current state into the config still sounds like it's depending on capturing state from the terminal and from all of the command line processes executing within it.

I would like to see allowing a definition of tabs to be opened on startup (much like other Windows terminal applications).

As far as restoring state, I accept what happens in the terminal to be volatile. If I want the output, I capture it with the commands executed.

It would be great if the Terminal could remember:

• List of tabs previously open and their current working directory
• List of commands previously ran in each tab (maybe last 10-20)

The actual contents of the buffer are not as important. When I restart my computer or accidentally close the Terminal app, I just want to get back to where I was as quickly as possible.

@nmoinvaz So those two might be generically impossible for the Terminal to solve.

• This thread:https://github.com/microsoft/terminal/issues/3158#issuecomment-628628120 (and the threads linked to it) has far, far too many details on why it's hard for the Terminal to get the working directory of the current process.
• As far as the list of commands run, that's the shell's responsibility, not the Terminal's. The Terminal doesn't know what is and isn't a command the user entered, it only knows about a stream of input it delivered to the client application. Notably, powershell/pwsh and bash already save the command history on exit. cmd is the one notorious exception, and it's not about to be updated to add that support, since that constantly burns us. See https://github.com/microsoft/terminal/issues/217#issuecomment-404240443.

To me, this is just like how browsers allow for reopening tabs. In fact, a terminal kinda is a browser. And while restoring the session might not be easy, restoring the path (and things like the tab title: https://docs.microsoft.com/en-us/windows/terminal/tutorials/tab-title) definitely ~is~ (things are never as easy as we users think) seems to be. For my use-cases, that would already be enough

Came here looking for a save last open tabs feature. I have been using Cmder which has a option to open all last closed tabs just like in a browser, it's a really handy feature let's you get to work where you left. I don't think it restores the history of each shell but just restoring tabs and the directory I was on is massive, just like a browser doesn't save the tab state and open the website like it was when last time you loaded (it refreshes it and opens it anew)

Me I mostly worry the tab colors 🙈😊

I created #7574 because I really think that the ability to save the current session is extremely important. There are many scenarios when I as a developer could have really benefited from it. Let's say you have several servers that need to be running in order for your app to work, and you're running them all from your command line through different tabs. It's a huge pain to open these tabs and then cd to each location every time you re-boot your computer or run updates.

What would be fantastic is the ability to save the state of the terminal (ie. the tabs and each of their locations) in a config. That way when you open your terminal, you can simply run a command like: terminal-open savedConfigName and your session will be restored.

@nmoinvaz So those two might be generically impossible for the Terminal to solve.

• This thread:#3158 (comment) (and the threads linked to it) has far, far too many details on why it's hard for the Terminal to get the working directory of the current process.
• As far as the list of commands run, that's the shell's responsibility, not the Terminal's. The Terminal doesn't know what is and isn't a command the user entered, it only knows about a stream of input it delivered to the client application. Notably, powershell/pwsh and bash already save the command history on exit. cmd is the one notorious exception, and it's not about to be updated to add that support, since that constantly burns us. See #217 (comment).

Regarding your first point, I believe simply being able to add the location to a config file manually would suffice. There are already a lot of settings that require manual editing of JSON, so this could very well be similar in that way. Perhaps an array of file system locations, and for each item in the array, a tab would be opened to that location.

@zadjii-msft, @DHowett guys, why does it take years to implement such simple, highly requested, important features? Don't you want this feature yourself?

Just like @nmoinvaz, @lukaseder, @danyalasif said above, no one asks for a complex system that messes with security, all we need is the ability to restore 2 basic properties for each tab onAppLoad event: tab: { cwd: String, commands: Array<string> }.

It literally takes only 15 minutes to figure out the logic + a few hours to implement it:

• Add 1 property to settings: settings: { startupAction: ('restore-last-state'|null) }
• Add 1 property to globalStore (where you store shared data): globalStore: { state: { tabs: Array<object> } }
• Add each tab 2 properties: globalStore: { state: { tabs: [{ tab: { cwd: String, commands: Array<string> } }] } }
• When the state of a tab changes, write the state to storage
• On app load, restore state from storage: get storageData.state and set it to globalStore.state

settings.json

settings: {
  startupAction: 'restore-last-state',
}

globalStore

globalStore: {
  state: {
    tabs: []
  }
}

storageData.json

{
 "state": {
    "tabs": []
  }
}

main

async function onAppLoad () {
  await initRestoreLastState()
  ...
}

async function initRestoreLastState () {
  if (settings.startupAction == 'restore-last-state') {
    globalStore.state = await getStorageProperty('state')
  }
}

async function getStorageProperty (key) {
  return JSON.parse(getStorageData())[key]
}

function onExecuteCommand () {
  saveTabCommand()
  saveTabCwd()
  writeStateToStorage()
  ...
}

function saveTabCommand () {
  get(stateCurrentTab).commands.push(event.command)
}

function saveTabCwd () {
  get(stateCurrentTab).cwd = getCwdPath()
}

function onNewTab () {
  addTab(event)
  writeStateToStorage()
  ...
}

function addTab () {
  let tab = {
    cwd: '',
    commands: []
  }
  tab.cwd = getCwdPath()
  globalStore.state.tabs.push(tab)
}

function writeStateToStorage () {
  if (settings.startupAction == 'restore-last-state') {
    // Avoid data corruption by writing data synchronously 
    // (file might get corrupted when multiple async processes writing at the same time)
    writeStorageSync('state', globalStore.state)
  }
}
...

@aleksey-hoffman we await your pull request! Thanks!

If it'll only take a few hours to implement, then I look forward to the PR 😉

Can't wait for your PR! @aleksey-hoffman

@DHowett @zadjii-msft @portexe seriously, with the sarcasm, guys? I meant it would take you a few hours because you already know the structure of the project and how everything works. Obviously it would take me longer than that, because I would have to learn the whole codebase in order to not mess anything up. I'm busy working on my own stuff and don't have time to learn how this project works.

I've laid out the whole restoring logic for you above. At first, it doesn't have to be any more complex than this. I use a similar system in my own app and I know that it works perfectly. Just add some error handlers and adapt the logic to your codebase, and it's done.

If you are just going to take questions and advice with sarcasm, I don't really wanna participate

@aleksey-hoffman it’s just... it’s generally considered poor form to find a feature request and gesticulate wildly about how “easy” it would be or how the maintainers are ignoring a common use case or something. We all want this feature! It’s just that we’ve spent the past year and change getting Terminal up and running and shippable before we come back and focus on the lower priority nice-to-haves.

Things like competent terminal emulation, performance and reliability, a UI people can understand, a robust configuration model, and a settings UI (popular request, even if some folks prefer JSON) have been prioritized ahead of other things like this and “well does the bold font actually look bold.”

In general, these decisions come down not to lack of knowledge or understanding of the problem space but of hours in the day for my team—which was up until quite recently only four people—to get work done. 😄

You've provided a gross over-simplification of how this feature would need to work. Let's break it down.

1. Figuring out the CWD of a process on windows is Hard. See #3158 and linked threads for an in-depth discussion of why.
2. Restoring just a list of entered commands - that's probably not what the user wants. The user wants the output in the buffer, probably in addition to the list of entered commands. However, the Terminal itself doesn't know what an "entered command" is. When the user has vim open, how would the Terminal differentiate that input from the input at the prompt line? The Terminal really can't which is why restoring command history is usually the responsibility of the shell application, not the Terminal.
3. The Terminal doesn't currently have a good way of storing runtime state like the globalStore you've mentioned above. We've been trying to avoid writing to the user's settings.json file, since that's been a minefield in the past, but that leaves us without a good way of storing non-user-facing application state. We've got #7972 open though tracking how we might want to do this in the future.
4. How do we handle restoring multiple windows? When you think about restoring tabs in a browser, it's usually about restoring all the windows with all the tabs, not just the tabs in a single window. We don't really have a "quit"-like feature that closes all the windows simultaneously, so you'd have to close the windows one at a time. When it comes time to restore the state of the window, that would just restore the last window you closed. We'd need some sort of cross-process communication to coordinate the saving/restoring of every window's state, which fortunately we're working on over in #5000
5. We haven't even gotten to mentioning pane state in addition to tab state. There's actually quite a bit of info we'll need per window/tab/pane - the terminal title, tab color, profile, split size and direction, etc.

All this together makes this feature a lot more work than just "a few hours", even for someone who's deeply familiar with the codebase. We're all very passionate about this project, and very excited to make it the best application we can, including this feature. Collaboration is a key part of that, and we're happy to accept community feedback and contributions. However, suggesting that something is something like this is "easy" is incredibly dismissive of all the hard work that is being done.

@DHowett I get it, we have to prioritize, but I'd argue this feature is way more important / time saving / desirable than the features like "is bold text really bold".

@zadjii-msft I don't know mate, I feel like you are overcomplicating this a bit.

You're saying the commands cannot be distinguished when you're using vim, but I'm talking about regular commands, not all people use vim, and if you do, there are other ways to distinguish real commands, e.g. create a keydown listener and record key presses. The functions that paste text (ctrl + v and via mouse) can also record the commands (e.g. extracting it from the clipboard API).

1. Isn't there a workaround for this? Why would it even need to extract CWD from somewhere if all commands are entered in plain text? The app could just parse all entered commands and check if any of them change the directory e.g. doesChangeDir(['cd *', '.. *'], command). If the command changes directory e.g. cd C:\test, write it to storage and, on the next launch, run cd C:\. Would that not work?

2. I disagree, I think most users just want to re-open the app, press up-arrow a few times and press enter rather than typing the path of the project directory and all the commands manually every single time. And the support for buffers can be added later.

3. I don't see why creating a shared memory store is a problem, Create 1 file with 1 class for storing shared data. Create a single instance and import it in every file that needs it. And as for the settings.json, I don't get the point. You are storing all user settings in settings.json but you don't want to store this particular user setting in there because there's already too many user settings? So opacity is good enough to be stored in there, but the launchAction is not, for some reason? That's what the file settings.json is for, is it not?

4. Okay, until you implement the IPC, create a function that sends all opened terminal processes taskkill /pid PROCESS_PID. Isn't that the same thing as closing the window manually by clicking on the X? If somehow Windows Terminal processes get closed forcefully even without the /f flag, then make this feature experimental and add a warning saying "all windows will get forcefully closed" before executing the function.

5. I don't see why it's a problem to store all the windows data on the drive and then restore it on launch. When a window is closed, do not delete that data so it can be restored on next launch. On the next launch, programmatically create all the windows, open all their tabs, and set all other data (panes, colors, etc):

I look forward to this PR, it'd make it way easier to launch my development environment(s).

Maybe it should be looked at from the direction of the workspace feature in VSCode?

Something along the lines of a window per "workspace", containing the different terminals and their history?

The best way we have to prioritize features is based off of community feedback. Right now, the best way of tracking that is through reactions on issues - not the best metric, no, but definitely something measurable. Just sorting by 👍's, this issue is below 12 other requests, 5 of which we're hoping to get done in 2.0. This one just simply didn't make the cut.

The simple fact of the matter is that we have to design a solution that will work in all cases. If only 20% of people use VIM, and our heuristic for detecting commands doesn't work when the user is using VIM, that means that the solution just won't work in 20% of cases. That's not a solution - that's a hack, and not something we'd be comfortable shipping to our users.

1. Even if we did track all the key events sent to a terminal, there's no way of knowing if a particular set of keystrokes resulted in a changing of directory. There's just far too many things that could result in the directory changing - there's cd, pushd/popd, Set-Location for cmd & powershell, or the user could be using something like #keep, or something like FAR manager where the user is using arrow keys (or even the mouse!) to navigate the filesystem. That's without considering the use of text editors, or scenarios like WSL, docker, or ssh, where the path inside the distro/container/remote machine aren't the same as on the outside. There's just no way of doing this heuristically from input. Fortunately, this isn't really an issue! Modern shells (read: not cmd.exe) all persist the command history themselves, so they'll all persist the history across reboots for you! The Terminal doesn't even need to get involved.
2. Okay, that's a fine opinion to have. See the above - the shell will restore the history for you, so that's why I don't think the command history is a valuable discussion to have.
3. It's not necessarily a problem, which is why the spec in #7972 is so incredibly straightforward. We don't want to be writing to the settings.json file, because in the past we've messed up the formatting of people's settings files, and people were greatly displeased with this, which is why we're introducing a second file to store this state. It's not terribly challenging, but this is just another piece of the puzzle that will take "a few hours" to implement, debug, write tests for, etc.
4. I'm pretty sure if we end up taskkill'ing all the Terminal processes, then none of them would have the chance to store the state of the open windows, entirely ruining any chance to restore the state of these open windows, and the whole point of the feature. I'd rather make sure that we implement this right the first time, rather than ship a half-baked hack and have people start taking a dependency on that behavior.
5. Before there's any sort of IPC way to coordinate between windows, how do we differentiate between the first launch of a Terminal window, and subsequent ones? Only the first launch should restore the state of the persisted windows, not the subsequent ones, right? You don't necessarily want each launch to pop up all the windows/tabs you had in the previous session (though, if you do, you might be interested in #756)

I'm looking forward to working on this, but it's probably not going to land until post 2.0. We're already going to have to work on serializing the state of a terminal tab as a part of #1256. Plus, we're already in the process of developing the IPC functionality as a part of #5000. So a lot of the hard problems that need to be solved before we can address this issue are already in progress!

(Frankly, most of this discussion is better off in #766 - this thread was supposed to specifically be the thread about restoring buffer contents, not window state)

@zadjii-msft well, then I suppose my estimates of "a few hours" were off, I based it on my own experience, but I didn't consider the fact that in my app I already had the IPC, the system for writing to storage, etc. and, for some reason, I assumed that all of those systems were also already in place in the Windows Terminal. Thanks for clarification.

@zadjii-msft This is a very interesting information about the likes measurement. Didn't know that one. https://docs.github.com/en/free-pro-team@latest/github/managing-your-work-on-github/sorting-issues-and-pull-requests

Just want to make folks aware of this other issue which is quite high in ranking already: https://github.com/microsoft/terminal/issues/766 using the sort thumbs up emoji https://github.com/microsoft/terminal/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc So I have voted now for both as until this is done terminal is a cool project, but until that I need to stick with other projects like Babun, CMDer or Conemu, as at least just reopening my last opened tabs is quite essential for me as well otherwise I always need to remember what was last opened which is quite difficult sometimes for me when it comes to having open more than 20 tabs.

For those looking for a workaround, I use an action like the following:

        { "command": { "action": "wt", "commandline": "new-tab --title OpenConsole cmd.exe /k #work 15 ; split-pane --title OpenConsole cmd.exe /k #work 15 ; split-pane -H cmd.exe /k media-commandline ; new-tab --title \"Symbols Script\" powershell dev\\symbols.ps1 ; new-tab -p \"Ubuntu 18.04\" ; new-tab -p \"microsoft/Terminal\" ; sp -V -p \"microsoft/Terminal\" ; sp -H -p \"microsoft/Terminal\" ; focus-tab -t 0" }, "name": "Good Morning" },

and then I invoke that from the command palette each morning after a Windows Update. That sets up the dev environment I usually use, with 4 tabs, and a bunch of panes in each. Customize as needed for your own profiles/environments. It's definitely not the same as "restore state" or "start with multiple tabs open", but it gets the job done.

@zadjii-msft I don't think the shells can really restore command history in this case meaningfully. They can save command history, but once multiple tabs/panes are active, the shell has no idea which pane is which. The naive thing they'll do is terminate all the shells and all append to .bash_history etc in random order, and on restore each pane will have command history from all the other panes in an undefined order. I don't see how to do this without ascribing each tab/pane some form of identity, having shells persist state related to that identity, and communicating that identity back to the shell on restore.

If that identity did exist though, it would allow shells to save/restore things like the environment, current directory, aliases or other shell state, etc, which seems useful.

Saving a command history per tab isn't even what I would expect, because this is a very shell dependent behaviour. Enough would be for me to recover the opened tabs plus ideally the saved last directory used in the particular tab. My assumption is that now that we have the environment variables WT_PROFILE_ID and WT_SESSION. That would be very enough already, as I am not expecting more from ConEmu.

So saving history per tab would already over scope of a beginning of this feature and in my opinion an enhancement already.

In my opinion writing this state to settings.json is not the right place. Better would be something like SQLite or some alternative.

Thank you all for kicking this discussion off.

@malxau You're not wrong here. We're not the first terminal emulator that's allowed multiple concurrent instances, however, so I'm thinking that this is more of an issue that shells just simply haven't found the right solution for yet. We've got the WT_SESSION variable that we use for terminal instances, though I wouldn't want anyone taking a dependency on anything like that, especially considering it wouldn't work in something like tmux. If there was a way of identifying unique terminal instances that worked across emulators and shells to help coordinate the restoring of history to the correct terminal instance, I'd be all ears 😄