steamtinkerlaunch STL stores non-config data in ~/.config

Currently, STL stores everything in XDG_CONFIG_HOME (usually ~/.config), however the XDG spec recommends that directory only be used for configuration files.

Cache files, that can be replaced at any time (e.g. the downloads directory) would be better off in XDG_CACHE_HOME (~/.cache).

Persistent data, that isn't config files, would be better off in XDG_DATA_HOME (~/.local/share). The various WINEPREFIX/compatdatas (e.g. for mo2/vortex) are probably a good example of non-config data.

This might seem pretty, but applications storing stuff in the correct places can often make administering a system a lot simpler.

Similar to when stl's config directory changed from ~/.config/stl to ~/.config/steamtinkerlaunch, this would be a "breaking" change, so would benefit from a major version bump and clear release notes, although most users should be completely unaffected.

Storing stuff in XDG_CACHE_HOME may also highlight any existing bugs where wiping stl's cache breaks stuff. In principle, every time STL uses a file stored in XDG_CACHE_HOME, it should first check if the file exists and if not, re-download (or re-generate) it.

Jun 06 '22 23:06 MattSturgeon

Nice and useful idea. I don't have much time, but try to help with the implementation when I find the time (assuming this was an offer)

Jun 07 '22 13:06 frostworx

I like this idea, but want to mention we should keep existing installations in mind. For things like MO2 which should go in $XDG_DATA_HOME we should think about having some way of moving these files around automatically if they're found.

Perhaps STL could check on startup to see if it's using the "old-style" file system (basically check what is present in ~/.config/steamtinkerlaunch, maybe like if [ -d "$STLCFGDIR/mo2" ]) and prompt a user. Something like this: "SteamTinkerLaunch v11.0 needs to run some extra configuration, please refer to the release notes for more information. Would you like to do it now? (This process is automatic but may take a while to complete)" With an option to accept or "Ask Again Next Launch." The line to refer to the release notes could have a link to the GitHub release page, and maybe we should also think about having an entry on the wiki on this once this feature is more finalised. I also recognise that this idea might take a considerable amount of testing.

Just my thoughts on how to make this kind of transition as simple as possible for an end user :slightly_smiling_face:

Jun 08 '22 23:06 sonic2kk

yeah, those migrations definitely need a (temporary) clean up function. (I think in the past I removed them too quickly, so I'd keep it for about two version bumps if this comes life)

Jun 09 '22 04:06 frostworx

hm, so no further discussion at all? 🤔

Jun 30 '22 13:06 frostworx

hm, so no further discussion at all? 🤔

Sorry I've been busy with work and other projects. Not sure if/when I'll have chance to work on this, but happy to if no one beats me to it 👍

Jun 30 '22 16:06 MattSturgeon

thanks for the reply. no problem, I just bumped (almost) all open issues today.

Not sure if/when I'll have chance to work on this, but happy to if no one beats me to it +1

hehe, same :)

Jun 30 '22 16:06 frostworx

Nothing which will or should happen completely at once, so I suggest we collect a proper list of files/dirs with the current and possibly the best position first, and then migrate one after another carefully.

[ ] ~/.config/steamtinkerlaunch/downloads → ~/.cache/steamtinkerlaunch/downloads (STLCACHEDIR)
[ ] ~/.config/steamtinkerlaunch/vortex/downloads → ~/.cache/steamtinkerlaunch/vortex/downloads (STLCACHEDIR)
[ ] ~/.config/steamtinkerlaunch/vortex → ~/.local/share/steamtinkerlaunch/vortex/downloads (STLDATADIR)
[ ] ~/.config/steamtinkerlaunch/wine→ ~/.local/share/steamtinkerlaunch/wine (STLDATADIR)
[ ] ~/.config/steamtinkerlaunch/proton→ ~/.local/share/steamtinkerlaunch/proton (STLDATADIR)
[ ] ~/.config/steamtinkerlaunch/mo2/→ ~/.local/share/steamtinkerlaunch/mo2 (STLDATADIR)
[ ] ~/.config/steamtinkerlaunch/games → ~/.cache/steamtinkerlaunch/games (STLCACHEDIR)
[ ] ~/.config/steamtinkerlaunch/depressurizer → ~/.local/share/steamtinkerlaunch/vortex/depressurizer (STLDATADIR)
[ ] ~/.config/steamtinkerlaunch/cheatengine → ~/.local/share/steamtinkerlaunch/cheatengine (STLDATADIR)
[ ] ~/.config/steamtinkerlaunch/backup → ~/.cache/steamtinkerlaunch/backup (STLCACHEDIR)
[ ] ~/.config/steamtinkerlaunch/logs → /var/log/steamtinkerlaunch ?

Please add missing ones and/or correct me if you disagree with one or another.

Depending on the source, we need to be especially careful when symlinks are involved, which need to be migrated as well.

Aug 03 '22 17:08 frostworx

hmm not so sure about ~/.config/steamtinkerlaunch/backup being moved to .cacheas it's sometimes deleted by the user and I can see backups as something that you may not want to delete so it may be better to put it in .local

same thing with ~/.config/steamtinkerlaunch/games/ not sure how I feel about that being in .cache while it's not something thats really that useful for the end user to go back to maybe it makes more sense for it to be in .local?

also noticed that downloads from some tools are being suggested for .cache and others for .local

that is fine as some downloads you don't really need anymore.

However it may just be easier to move the whole ~/.config/steamtinkerlaunch/downloads/ to ~/.local/share/steamtinkerlaunch/downloads/

maybe if anything some none essintial stuff can latter be moved to .cache

such as :

~/.config/steamtinkerlaunch/downloads/cheatengine

~/.config/steamtinkerlaunch/downloads/d3dcompiler_47

~/.config/steamtinkerlaunch/downloads/reshade

and instead of being ~/.cache/steamtinkerlaunch/downloads/reshade for example it can be ~/.cache/steamtinkerlaunch/reshade/

I think that makes more sense as .cache kinda already implies temp downloads and stuff like that

As for logs, I think they should be moved to ~/.local/steamtinkerlaunch/logs/

i didn't see you mention ~/.config/steamtinkerlaunch/meta/? think that can go in ~/.cache/steamtinkerlaunch/meta

Aug 03 '22 23:08 zany130

so it would look something like this

home
└── user
    ├── .cache
    │   └── steamtinkerlaunch
    │       ├── cheatengine
    │       ├── custom-fallback.png
    │       ├── d3dcompiler_47
    │     
    │       ├── protonlist.txt
    │       ├── protonrating
    │       │   ├── 1198090.json
    │       │   ├── 253230.json
    │       │   ├── 31337.json
    │       │   ├── 391540.json
    │       │   ├── 524220.json
    │       │   ├── 991270.json
    │       │   └── 992490.json
    │       ├── reshade
    │       ├── specialk
    │       └── temp
    │           └── dxvkcfg.txt
    ├── .config
    │   └── steamtinkerlaunch
    │       ├── collections
    │       ├── default_template.conf
    │       ├── dxvk
    │       ├── gamecfgs
    │       │   ├── customvars
    │       │   ├── id
    │       │   └── title
    │       ├── global.conf
    │       ├── guicfgs
    │       ├── hide
    │         ├── meta
    │       │   ├── eval
    │       │   │   ├── id
    │       │   │   ├── old
    │       │   │   └── title
    │       │   ├── id
    │       │   │   ├── custom
    │       │   │   └── general
    │       │   └── title
    │       │       ├── custom
    │       │       └── general
    │       ├── menublock.conf
    │       ├── menusort.conf
    │       ├── tweaks
    │       │   ├── cmd
    │       │   ├── sbs
    │       │   └── user
    │       └── url.conf
    └── .local
        └── share
            └── steamtinkerlaunch
                ├── backup
                │   ├── exclude
                │   └── steamuser
                │       ├── id
                │       └── title
                ├── cheatengine
                │   └── Cheat Engine 7.4
                ├── compatdata
                ├── downloads
                │   ├── proton
                │   │   └── custom
                │   ├── shaders
                │   ├── steamgriddb
                │   │   └── backup
                │   └── wine
                ├── games
                │   ├── appinfo
                │   ├── desktop
                │   ├── desktopfiles
                │   ├── header
                │   ├── icons
                │   │   ├── ico
                │   │   ├── png
                │   │   └── zip
                │   └── pev
                │       ├── peldd
                │       │   ├── id
                │       │   └── title
                │       ├── pepack
                │       │   ├── id
                │       │   └── title
                │       ├── peres
                │       │   ├── id
                │       │   └── title
                │       ├── pescan
                │       │   ├── id
                │       │   └── title
                │       ├── pesec
                │       │   ├── id
                │       │   └── title
                │       ├── pestr
                │       │   ├── id
                │       │   └── title
                │       └── readpe
                │           ├── id
                │           └── title
                ├── lang
                ├── logs
                │   ├── dxvk
                │   ├── gamelaunch
                │   │   ├── id
                │   │   └── title
                │   ├── playtime
                │   ├── proton
                │   │   ├── id
                │   │   └── title
                │   ├── steamtinkerlaunch
                │   │   ├── id
                │   │   ├── lastrun.txt
                │   │   └── title
                │   ├── vkd3d
                │   └── wine
                ├── mo2
                │   └── compatdata
                ├── proton
                │   └── custom
                ├── vortex
                │   ├── compatdata
                │   └── downloads
                └── wine

I moved ratings from ~/.config/steamtinkerlaunch/downloads/proton/ratings to ~/.cache/steamtinkerlaunch/protonratings/ as it makes more sense there

so basicly

.cache

[ ] ~/.config/steamtinkerlaunch/downloads/cheatengine → ~/.cache/steamtinkerlaunch/cheatengine
[ ] ~/.config/steamtinkerlaunch/downloads/d3dcompiler → ~/.cache/steamtinkerlaunch/d3dcompiler
[ ] ~/.config/steamtinkerlaunch/downloads/proton/rating/ → ~/.cache/steamtinkerlaunch/protonrating/
[ ] ~/.config/steamtinkerlaunch/downloads/reshade/ → ~/.cache/steamtinkerlaunch/reshade/
[ ] ~/.config/steamtinkerlaunch/downloads/specialk/ → ~/.cache/steamtinkerlaunch/specialk/
[ ] ~/.config/steamtinkerlaunch/temp/ → ~/.cache/steamtinkerlaunch/temp/
[ ] ~/.config/steamtinkerlaunch/protonlist.txt → ~/.cache/steamtinkerlaunch/protonlist.txt/
[ ] ~/.config/steamtinkerlaunch/downloads/custom-fallback.png → ~/.cache/steamtinkerlaunch/custom-fallback.png

.local

[ ] ~/.config/steamtinkerlaunch/compatdata/ → ~/.local/share/steamtinkerlaunch/compatdata/
[ ] ~/.config/steamtinkerlaunch/lang → ~/.local/share/steamtinkerlaunch/lang
[ ] ~/.config/steamtinkerlaunch/downloads → ~/.local/share/steamtinkerlaunch/downloads
[ ] ~/.config/steamtinkerlaunch/vortex → ~/.local/share/steamtinkerlaunch/vortex/downloads
[ ] ~/.config/steamtinkerlaunch/wine→ ~/.local/share/steamtinkerlaunch/wine
[ ] ~/.config/steamtinkerlaunch/proton→ ~/.local/share/steamtinkerlaunch/proton
[ ] ~/.config/steamtinkerlaunch/mo2/→ ~/.local/share/steamtinkerlaunch/mo2
[ ] ~/.config/steamtinkerlaunch/games → ~/.local/share/steamtinkerlaunch/games not to sure maybe .cache does make sense?
[ ] ~/.config/steamtinkerlaunch/depressurizer → ~/.local/share/steamtinkerlaunch/vortex/depressurizer
[ ] ~/.config/steamtinkerlaunch/cheatengine → ~/.local/share/steamtinkerlaunch/cheatengine
[ ] ~/.config/steamtinkerlaunch/backup → ~/.local/share/steamtinkerlaunch/backup
[ ] ~/.config/steamtinkerlaunch/logs → ~/.local/share/steamtinkerlaunch/logs/

EDIT: didn't relize meta has config files for if you want to do a custom steamfirsttimerun script so that should stay in .config

Aug 03 '22 23:08 zany130

thanks for your suggestions. Although being clean and nice, looking at it, it might be even more confusing for some users than the single config directory though. Maybe the generally better approach would be to make those the corresponding directories simply overridable in the config, and let the user decide what they want (so 99% leaving the default settings untouched)

Aug 04 '22 04:08 frostworx

I think that makes more sense as .cache kinda already implies temp downloads and stuff like that

Not sure I agree with that. Temporary stuff should go in /tmp or similar. XDG_CACHE_HOME is persistent (non-volatile)... It isn't for temporary files, it's for files that are non-essential. Files that can be cached to increase performance, but can be replaced automatically if they disappear.

i.e. we can assume the files will be there for a while, but we shouldn't rely on it. Anything that STL can automatically replace probably belongs in .cache. Anything that is only needed briefly/occasionally and is cheap to replace could go in /tmp, although I don't think STL needs this.

Anything that can't be generated automatically is probably either data (.local/share) or config (.config).

hmm not so sure about ~/.config/steamtinkerlaunch/backup being moved to .cache [...] it may be better to put it in .local

Agreed, backups are "essential" data so shouldn't go in cache.

directories [could be] overridable in the config, and let the user decide what they want (so 99% leaving the default settings untouched)

Sure, config_dir, data_dir, cache_dir could make sense as config options. As long as they fallback to XDG spec when empty/unset. Could have additional more specific options too if needed (and not too complex) like download_dir for example...

As a stretch goal, it'd be nice if these options supported XDG env vars even when unset in the global environment. That way the user could reliably use variables in the config or they could be used in default values.

Aug 04 '22 05:08 MattSturgeon

Sure, config_dir, data_dir, cache_dir could make sense as config options. As long as they fallback to XDG spec when empty/unset

Sure, as in https://github.com/frostworx/steamtinkerlaunch/commit/ea0ec14d68eb849629d86ee3f2ca70df95981e55

Apparently I want to avoid spending much time into this feature. Of course it is nice to have for a few people, but the migration is much more complicated than just switching some paths. I'll think about a sane solution shortly.

Aug 04 '22 05:08 frostworx

I'm going to make a new release shortly and won't work very much on steamtinkerlaunch starting from then for various reasons. So sorry, I won't add that feature anymore. I will still stick around and occasionally add some things, but generally, I hope that the community rises with contributing PRs to keep the project alive. (this one will be pretty much work when every obstacle is avoided, so I'd perfectly understand if it won't be implemented at all)

Aug 07 '22 06:08 frostworx

I don't know if anyone is actively working on this but just a heads up if anyone ever does take a stab at this, ProtonUp-Qt's STL uninstaller will need to be updated. I don't think it's worth submitting anything to ProtonUp-Qt yet but in future, when all of the folders for the new directory structure are ironed out, the ProtonUp-Qt uninstallation logic will need updated to remove the folders.

So just as a note for the future if this ever does come to fruition, to keep this in mind before merging :smiley:

Sep 22 '22 16:09 sonic2kk

Thanks for the heads-up! (I do not work on it and likely won't)

Sep 22 '22 16:09 frostworx

This would be a behemoth of a task to be honest and while I am very much in favour of following the XDG spec, as it currently stands I am not super interested in doing it myself.

However if anyone wants to pick this up and work on it I would be more than happy to review and help out where possible with some knowledge. Some of the work was already started in ea0ec14d68eb849629d86ee3f2ca70df95981e55 a while ago but I'm not sure how comprehensive it really is.

Also, various parts of the wiki would need updated off the back of this change. ProtonUp-Qt support and possibly Steam Deck (and maybe even Flatpak) support would need to be at least reviewed off the back of this. On top of this, there would probably need to be a migration path, which would be very delicate and need heavy testing.

Even though I don't want to do this myself, anyone passionate, willing and knowledgeable on what changes to make to best follow the XDG spec are very welcome to come forward and work on it. A fair warning, review on it could be lengthy and testing even moreso, but having this implemented would be a great change in my opinion. At the time of writing though, I have other things I would like to work on :-)

Nov 02 '22 01:11 sonic2kk