Add support for roaming settings.json or storing it elsewhere

[patriksvensson]

Description of the new feature/enhancement

I have three different computers that I use for work. I keep my PowerShell profile in a GitHub repository and dot source it in my local PowerShell profile. That way I will only need to do a git pull on my profile repository to get all changes propagated on each computer.

It would be great if I could do something similar with my Windows Terminal settings by simply stating in my local settings.json file that the "real" settings could be found somewhere else.

(It could even be that the loaded profile acts like a base for the settings on the computer so things could be overridden, but that is a completely different issue that remains to be opened)

I'm aware that I might have overlooked something here, but would love to start a discussion about this since I suspect that I'm not the only one with this "problem".

Proposed technical implementation details (optional)

{
    "$schema": "https://aka.ms/terminal-profiles-schema",
    "globals": {
        "loadProfileFrom": "C:/Source/git/profiles/terminal_profile.json"
    }

[brandonh-msft]

any reason we can't just use the UWP Roaming data store? https://docs.microsoft.com/en-us/windows/uwp/design/app-settings/store-and-retrieve-app-data#roaming-data

I know this isn't a true UWP app, but Centennials can still use the UWP APIs, right?

[DHowett-MSFT]

We actually used to roam your settings using those exact APIs, but it caused us way more headache than it was worth. There’s a couple things we need to nail before we consider turning roaming back on.

[brandonh-msft]

yeah was just coming back to edit.

looks like per here: https://docs.microsoft.com/en-us/windows/apps/desktop/modernize/desktop-to-uwp-enhance and here: https://docs.microsoft.com/en-us/windows/apps/desktop/modernize/desktop-to-uwp-supported-api

we can't simply use RoamingFolder as I'd hoped? In lieu of this, then, @patriksvensson's approach is probably the best we'll get and users - well I, for one - could simply point at a OneDrive (or similar) location.

[zadjii-msft]

I'm more using this issue as a tracker for "add a way to import settings from other paths" than a "re-enable settings roaming". Using RoamingState for storing the settings was a pain. However, I think that we all agree adding support for importing settings from another location is a good idea, but one that needs careful thought and preferably a spec to go with it.

This is even a scenario referenced in the original cascading settings spec, as something we could consider in the future.

#1770 was the original bug tracking disabling roaming settings. Note the pile of other issues that were dupe'd to it.

---

April 15, 2022 edit:

I'm sneaking in here to add some notes about what re-enabling this again might look like. I'm reusing this old comment to avoid pinging folks till I have a plan.

What went wrong the first time

Plan for this time

• [ ] When we load the settings, look first in roamingstate, then fall back to localstate.
• [ ] Write the settings back out to roaming state.
• [ ] Should state.json's get written to roaming? Quite surely not.
  • ⚠️ if they've deleted a WSL profile from their settings, then the file gets roamed, and that distro is on the new machine (where there's not yet a state.json), the profile will come back to their settings file. It'll also then get roamed BACK to the machine that had originally deleted it.
• [ ] If we load from roaming, and see that there's a dynamic profile in there that we don't have locally, prompt the user to install it.
  • How does this combine with the "whoops we couldn't find your default profile" warning?
  • Make it an infobar maybe?
  • Can we easily just winget install {pwsh}, winget install {pwsh preview} in a new tab? that would be crisp
• [ ] What do we do about fragments?
• [ ] What how do we deal with symlinks? Do we need to do anything?
  • I suppose, if a user today has their localstate symlinked elsewhere, then they want the file to live elsewhere. They want it written to the elsewhere.
  • ⚠️ If we add a new symlink to the roamingstate folder, will it roam the contents of the link target to the roamingstate on a new machine?
  • If they add a symlink in roaming state, that'll probably get blown away if the settings roam back, yea?
    • It's probably okay to let them just symlink in LocalState though, and that'll prevent us from loading the roamed file.
  • Is just not roaming settings for folks with a symlink a reasonable UX? "I've taken things into my own hands"
• [ ] more....

Other things to keep in mind

• "Store settings elsewhere" - like, load additional files into the settings. IIRC my original imagination for this was something like "include": [ "{path-to-file.json}" ] as a global setting that would load cascade additional paths.
  • This was originally #2303, but we thought fragments sufficiently addressed that. It might not have.
  • Surely, the paths for additionally included settings files won't work on the machine they were roamed to. May be okay to silently ignore those failures though.
• #4566
  • in general I hate this. Loading settings based on the value of an env var that's set seems fragile or prone to not working as expected, esp. given the number of issues we've had with env vars in the past.
• #6687
  • Much less fragile than the above, but still comes with issues. We'd have to make sure that the window that's started with those settings then only writes back to that file?
• Are there other roaming types that we might want to deal with? If the user has multiple MSA's logged in, is that an issue? Something like the "merge" mentioned here
• #12807
  • We'll need to document this, cause it'll never work.
  • This will obviously be annoying.

[patriksvensson]

@zadjii-msft What's the process for creating a spec for something? Do you have an RFC-process or similar?

[zadjii-msft]

@patriksvensson We don't have an official "RFC" process, but we do have a semi-formal spec review process. This helps us ensure the entire team is on-board with more complicated features, and for scenarios like this one, some of the more complicated edge cases have been thought out. I'd refer to the following:

• Contributing-to a Spec
• The spec template
• completed specs
• all spec PRs

[patriksvensson]

@zadjii-msft Would it help if I (or someone else) drafted up a spec to get the ball rolling?

[zadjii-msft]

I think that would be very helpful :)

[jpetkau]

As a workaround for this, it's possible to make profiles.json a symlink to a shared settings file in Roaming or OneDrive or Dropbox or whatever.

This mostly works, except terminal doesn't notice changes to the actual file. It only watches the symlink. So hot-reloading of changes doesn't work.

Of course roaming just magically working would be better.

[chwarr]

There's a related issue in this space, #4566 "Consider adding a WT_PROFILE env var pointing to settings file for 1.0 release".

It proposes setting the environment variable WT_PROFILE to the path of the user's settings file. Copying a comment that I made on that here:

My expectation of a variable like WT_PROFILE would be that I could [also] use it to change where Windows Terminal reads user settings from. ...

Based on how other, similar software works, I think it would make sense if Windows Terminal only read this environment variable at process start up and then monitored the path for changes. It would also be reasonable for Windows Terminal to detect changes to the environment "template" by handling WM_SETTINGCHANGE notifications. (Though, there are some subtle details to consider about when changes are applied. Lots of thoughts in #1125, "Feature Request: Terminal should listen for the WM_SETTINGCHANGE for environment variable updates".)

[patriksvensson]

If anyone is interested in an interim solution to this, I've written some instructions here: https://www.patriksvensson.se/2019/12/roaming-profiles-with-windows-terminal

[DavidGamba]

If the file watcher that monitors profiles.json can detect a symlink and watch the target instead many users would easily backup their config in multiple ways and still get the full benefits of hot reload of the config.

[DavidGamba]

If the file watcher that monitors profiles.json can detect a symlink and watch the target instead many users would easily backup their config in multiple ways and still get the full benefits of hot reload of the config.

[zadjii-msft]

From @MartinSGill in #5638

Proposed technical implementation details (optional)

The common linux practice of using a settings.d style folder and loading all files in that folder seems like a good approach to me.

* Settings.json defines an "include" directory (environment variables permitted)

* Settings loads all files in that folder in alphabetical order.

* Settings adds/overrides settings based on loaded files.

* Additional files would all have the same structure/schema as the default settings.json

* [Extra Credit] Support multiple folders, allow shared/user settings.

[MartinSGill]

Advantages I see for my proposed implementation:

• it would support tools such as chef/ansible to install "features".
• It would also allow sharing of themes/keyboard-maps etc without upsetting the current settings. Add the file you want to try out to load last, try it out, if you don't like it, just delete the file.
• It would allow you to make profiles portable (e.g. CloudDrive/Dropbox)

[scottwillmoore]

As a non-ideal workaround for any that want to sync settings.json into their dotfiles while preserving the ability to hot reload, you can a hard link instead of a symbolic link. The file will remain synchronised with your dotfiles and a hard link will still trigger the filesystem events required to trigger a hot reload provided that you edit settings.json from the original path (just press Ctrl+, in the terminal).

An alternative approach that I have read somewhere else (but cannot remember where...) is to create a junction or symlink for the LocalState folder (the folder that contains settings.json) and I believe this will detect and trigger a hot reload in all cases.

[patricknelson]

An alternative approach that I have read somewhere else (but cannot remember where...) is to create a junction or symlink for the LocalState folder (the folder that contains settings.json) and I believe this will detect and trigger a hot reload in all cases.

I use Link Shell Extension for this. Also, I just tried your recommendation (dropping LocalState folder as a symbolic link) and it worked for me! Initially it didn't work since I only symlinked the settings.json. Now I can save it in Google Drive, edit it from either location and it'll update live.

[Firehawke]

I'll second Link Shell Extension. It's a really useful tool for a lot of different purposes. I don't want to get too far off topic, but I'll just briefly add that I used junctions via LSE to have Steam stuff on a non-C drive before Steam supported multi-drive libraries. I still use a lot of junctions to keep large parts of my system config on private cloud storage.

[bavis-m]

It appears like a Junction point to LocalState allows settings.json to dynamically update, but not a symlink (either to LocalState or settings.json). Can anyone else confirm these behaviour? A couple of replies here seem to suggest a symlink will work (which I would like, since I can't put a junction to my config repo which I usually have checked out in WSL).

[razamatan]

so i'm here b/c i just switched to wsl2, and i have a setup where my dotfiles are for linux and replicated by git. therefore, my dotfiles live in wsl. under wsl1, since the wsl rootfs was readily accessible via windows, i had my %LOCALAPPDATA%\Packages\Microsoft.WindowsTerminal_8wekyb3d8bbwe\LocalState\settings.json linked to the file in wsl (dir output: <SYMLINK> settings.json [\\wsl$\Ubuntu\home\razamatan\.dotfiles\settings.json]).

since wsl2 switched to vhd's, if i try to open terminal w/o starting wsl to load the vhd, i get the "Encountered errors while loading user settings" error dialog. i have to remember to always start wsl to allow wt to be able to read from that symlink.

it would be great to load wsl if trying to access any symlink that points to wsl paths. don't know if this belongs to wsl (https://github.com/microsoft/WSL/issues/6524) or wt as an issue, but posting here just in case.

*edit: just tried making settings.json a junctioned symlink, but that never works, even w/ wsl loaded...

[ghost]

Add Cloud sync support with our Microsoft account.

[thernstig]

If anything, I think to be best-in-class it should follow how VS Code does this. FIrst let a user have User settings as well as Machine settings (which takes precedence over User settings when on a particular machine, but both should be synced still for backup purposes). Then add different Authentication providers that one can connect to, to sync the settings automatically. Examples of such providers: Github account, Microsoft account.

Syncing is then done seamlessly in the background when logged in. You can get history of syncs. Etc.

[Falven]

If anything, I think to be best-in-class it should follow how VS Code does this. FIrst let a user have User settings as well as Machine settings (which takes precedence over User settings when on a particular machine, but both should be synced still for backup purposes). Then add different Authentication providers that one can connect to, to sync the settings automatically. Examples of such providers: Github account, Microsoft account.

Syncing is then done seamlessly in the background when logged in. You can get history of syncs. Etc.

Any ideas if this is being implemented? I see the thread being closed and reopened multiple times but no work item linked or anything.

[zadjii-msft]

I see the thread being closed and reopened multiple times but no work item linked or anything.

???

This thread has remained open since it was first filed. Maybe you posted on the wrong thread?

We're not working on this currently, nor do we have plans to do this in the 2.0 timeframe. The original roaming settings were a pain, but easy to implement. The VsCode method of roaming though sounds like quite a bit more work. Better for sure, but not something we're about to get to any time soon.

[Falven]

We're not working on this currently, nor do we have plans to do this in the 2.0 timeframe. The original roaming settings were a pain, but easy to implement. The VsCode method of roaming though sounds like quite a bit more work. Better for sure, but not something we're about to get to any time soon.

Why not? It could be developed with community support. The source code to do this is already available in the VS Code repository, it is production-tested, and it includes tests... As an aside I would hope that at a minimum WT should have extensibility as a major milestone in the roadmap to allow people to develop plugins such as this.

[zadjii-msft]

I would LOVE to have community help implementing this!

Just because we don't have time on the schedule to design/implement a feature ourselves, doesn't mean we aren't willing to help the community with such a feature. I just don't want to commit to anything, if we haven't explicitly allocated engineers to finishing it ☺️.

<aside> Yep, extensions is totally something we've been working on. #4000 is tracking that. That's probably the big 3.0 milestone feature. We still need to get some OS support worked out to help us - it's a little bit trickier to load actual native code extensions rather than just blobs of javascript 😆

[awakecoding]

I'm taking a chance here to suggest exactly what I intend to patch on my own builds to fit my needs: modify GetBaseSettingsPath to check for an environment variable "WT_BASE_SETTINGS_PATH" and use it if it's present instead of the default path. I really don't need anything more than that, as I would just launch Windows Terminal with different environment variables to make it point to different directories with injected settings.

Is there any chance that such a small change could be considered? I don't care about the environment variable name, just that we can use an environment variable (or command-line parameter) to override the base settings path.

[DHowett]

I took a crack at writing up the open questions about how a configurable base (be it a command line argument, environment variable, etc.) would work over at https://github.com/microsoft/terminal/issues/6687#issuecomment-650382579 :smile:

[awakecoding]

I took a crack at writing up the open questions about how a configurable base (be it a command line argument, environment variable, etc.) would work over at #6687 (comment) 😄

This looks good to me, is there a branch with a prototype for the suggested change? Also, while settings.json is the primary file used by Windows Terminal, would it better to override the directory containing settings.json rather than the full path to settings.json, such that if Windows Terminal decides to store more files in the future, we wouldn't need to modify it again to override additional paths? Overriding the base path containing all files appears to be the safest option long term.

[patricknelson]

Thanks for calling that out @DHowett

Hmm… since some of us want to synchronize profiles and others want the ability to parameterize via CLI which profile to use, have we considered separating this into two components? i.e.

1. Directory to load profiles from
2. The actual JSON file to load (must be a file, not a path and cannot be relative)

I say this since it might help to abstract away our concerns of allowing it to be roaming whilst also facilitating parametrized variability without being too tightly bound to the actual path. For example, both wt -c profile1.json and wt -c profile2.json would still work and are separate from where we decide to put them. This also suggests of course that, if the directory itself isn’t provided as a parameter as well (or some kind of env var, cringe I know) that you’ll have to devise a sane default. Presumably it’d still be in the current directory where this file is already stored and/or also configurable at some higher global level, wherever that may be.

I’m just trying to offer a concept that might help reconcile these two features amicably without it getting too complicated later on. 🤞