tmuxp

tmuxp copied to clipboard

Hopefully this isn't off topic here; Does tmuxp provide an alternative to TPM/tmux-resurrect? I'd like something where all the programs aren't written in bash, and tmux-resurrect + tmux-continuum has some problems.

tmuxp's --freeze option sounds good, but at least restoring scrollback, and at least maybe for bash, somehow having per-pane history files would be nice. (I managed to get this working relatively ok for the above plugins once....)

Is tmuxp / libtmux appropriate for attempting a better reimplementation of these? Thanks!

I'm a heavy, stateful user of the terminal and I have a lot of semi-finished things open in tmux a lot. It's really terrible when a reboot/battery depletion/crash makes me lose things. :( https://en.wikipedia.org/wiki/Persistence_(computer_science)#Orthogonal_or_transparent_persistence

I haven't used tmux-resurrect. To my knowledge it also freezes processes, right?

tmuxp freeze works a bit differently. tmuxp doesn't preserve processes (though I guess we could add an ability to interoperate with tmux-resurrect?). tmuxp creates a new tmux session via commands to tmux and starts new shells.

If someone wants to PR optional support for that or anything in tmux-plugins, it's fine with me.

No, to the best of my knowledge resurrect has an understandably rather naive restore model. I think it checks the running process via ps and then it uses some save/restore scripts to handle individual processes. I think there's some build in vim integration, but not arbitrary processes.

• but don't quote me on this.

My main priority would be restoring bash history and scrollback - the approach of resurrect should be possible to port for starters (assumption). I figured via adding appropriate functionality to the freeze sub-command.

I'm working on restoring scrollback, I have dumping to a file minimally working. Would per-pane config file entries like this be appropriate, or do you have any better ideas?

"history": {    
            "shell_history" : file,
            "scrollback" : file
            }

My current idea is both of those file fields enable the history type for the pane if they are present, otherwise it's not used.

An alternative is to just have a global option for a dump directory.

The global dump directory option involves less file and editing noise - but this probably isn't relevant because the json file is basically autogenerated? The per-pane version is more flexible if someone wants. Both could also be done but then that needs a reasonably defined precedence semantics. (Global option unless stated otherwise?)

tmux-resurrect currently seems to have some thoughts about deprecating the shell history function due to it's "invasivity", would you accept the functionality?

I think the way it currently works for resurrect is that they autotype save/restore commands into the pane. I have some more ideas like setting the history file environment variables or attaching gdb to the bash process trees (similar to https://www.commandlinefu.com/commands/view/11427/dump-bash-command-history-of-an-active-shell-user.), but none of these seem like particularly good solutions, but I think the end result would still be nice even if it only works most of the time.

Do you have some way we could discuss the design a bit? - and then I will summarize here for posterity.

@deliciouslytyped

Up front - there are some aspects of this where tmux-resurrect overlaps with tmuxp kinda. On the tmux-resurrect readme, they mention migrating from tmuxinator (a similar thing to tmuxp). I have too much on my plate at the moment to go into resurrect in depth.

About chat: I got your email about that. Not enough to talk about yet, but in case it comes up in the future: I sent a mail to gitter.im asking for help getting orgs setup for tmux-python (and vcs-python). I tried setting it up a long time ago.

Regarding design:

I propose we find a way to add hook functionality to tmuxp.

We could use python import strings to allow arbitrary functions to execute python code, and we can pass libtmux objects inside.

plugins:
  - 'my_tmuxp_plugin.plugin.TmuxResurrect'
session_name: my session
windows:
- window_name: my window
  layout: tiled
  shell_command_before:
    - cd ~/                    # run as a first command in all panes
  panes:
    - echo pane
    - echo pane

Assume a git repo my_tmuxp_plugin with a directory my_tmuxp_plugin inside it, and a pypi package called my_tmuxp_plugin:

my_tmuxp_plugin/plugin.py:

class TmuxResurrect:
    def before_workspace_builder(self, session):
        # On blank workspace, after session created, but before any windows/panes/commands entered

        # technically, it'd be possible to do any kind of subprocess call here as well

    def before_script(self, session):
        # access to libtmux session object in tmuxp load

   def reattach(self, session):
       # if session_name already live, before reattaching

   def on_window_create(self, window):
        if window.window_name == 'my window':
            # do stuff

   def after_window_finished(self, window):  # after commands stuff commands finally entered
       pass

The thing with tmux resurrect though is 1.) they need to have it installed via TPM and/or have it in .tmux.conf.

Any ideas? Is my plugin idea a way you think tmux-resurrect / tmux plugins could be integrated into tmuxp?

@deliciouslytyped I should also add I don't have much time to add / maintain features to tmuxp. I'm very lucky I had time to do some winter cleaning yesterday to CI *

I think the API thing makes sense since it'd allow linking into the libtmux under the hood and and pretty much doing anything. It'd also open up an area of community contribution and perhaps integration with other tmux plugins.

* I'm looking for maintainers actually https://github.com/tmux-python/tmuxp/issues/290 if you like working with python and tmux

I'm pretty partial to IRC + a logging bot, but hey, it's your project. :D

I'm going to have to read over that again. I don't have any good ideas offhand.

My ability to work on things is also massively fluctuating and I'm not a huge fan of how my code usually ends up looking, so I'm not sure I could integrate nicely into the style of tmuxp. I kind of hoped I could get the core functionality working and then get some guidance on rough edges.

This isn't necessariy an ideal time for me to rearchtect the core of tmuxp... :sweat_smile:

@deliciouslytyped

This isn't necessariy an ideal time for me to rearchtect the core of tmuxp... 😅

I could stub out a plugin system to make libtmux objects available. Since I'm very familiar with the codebase, import strings, and libtmux. I could add tests and write a guide on making plugins.

What I can't address is writing tmux plugins themselves. Does that make sense?

Sure.

@deliciouslytyped The cool thing about this is if a plugin is added, it could/would be passed the Session, Window or Pane object being adjusted.

That is a very powerful API. Basically it means any python code could run, and they'd have access to libtmux.

I created an issue at #530 for a plugin system.

For my current iteration of resurrect I just have this small snippet added to the inner pane loop of freeze() (using a patched verison of libtmux that has a more thorough capture_pane), and I haven't written any code for restoring yet.

           import os
            if "TMUXP_DUMP_PATH" in os.environ:
              dump_path = os.environ["TMUXP_DUMP_PATH"]
              dump_path = os.path.dirname( dump_path + "/" )
              try:
                os.mkdir(dump_path)
              except FileExistsError:
                pass
            if dump_path is not None:
              print("Dumping %s" % p)
              with open(os.path.join(dump_path, "%s-%s-%s" % (p["session_id"] , p["window_id"], p["pane_id"])), "wb") as f:
                val = p.capture_pane()
                for l in val:
                  f.write((l + "\n").encode("utf-8")) #TODO whats the correct way to handle the encoding? check some shell env var?
              print("Done dumping %s" % p)

@deliciouslytyped Can you give me the git diff output of it?

Also, are there any other places inside of tmuxp you would find it helpful to "hook" into?

Well, the problem is I don't know yet. The only things I can think of off the top of my head is that I'm going to need access to the above mentioned or similar config options, yeah? - or have the config options trigger my code? IDK - the usual possible variants on plugin infrastructure ...

So, both read access for restore, and write access for freeze.

I don't know what a good way to do this would be. A stack of pre-post hooks for config read-in and the config passed as a mutable variable (ugh) to every hook?

I'll try to do some WIP PRs in a bit for libtmux and tmuxp, I guess.

@deliciouslytyped Proposal: If you create an MVP and show a git diff (it's okay if its hacky), I can propose a way it'd be doable as a plugin #530

I'll try to do some WIP PRs in a bit for libtmux and tmuxp, I guess.

Yes, show a git diff for any changes to tmuxp/libtmux it takes to get tmux-resurrect working. It can be hard-coded. Also you can make a pull request for it and make it editable. Not sure if it's merge-able, but I can technically edit it directly then

No promises but I'm going to be on a train for a few hours so with any luck I can throw something together. - modulo network access.

I'm not sure you can help with this much information, but this is just a simple restore, any idea about the problem?:

$ tmuxp load -S ./socket 3.json
[Loading] /run/user/1000/tmp.kSvepSlq6I/3.json
Traceback (most recent call last):
  File "/nix/store/hnnz9g0kswahg630b4rwxh00l8bw2pmd-tmuxp-1.5.4/bin/.tmuxp-wrapped", line 9, in <module>
    sys.exit(cli.cli())
  File "/nix/store/lma5qkl6r8vqrbhs7w202fxbrxd1c7fa-python3.7-click-7.0/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/nix/store/lma5qkl6r8vqrbhs7w202fxbrxd1c7fa-python3.7-click-7.0/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/nix/store/lma5qkl6r8vqrbhs7w202fxbrxd1c7fa-python3.7-click-7.0/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/nix/store/lma5qkl6r8vqrbhs7w202fxbrxd1c7fa-python3.7-click-7.0/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/nix/store/lma5qkl6r8vqrbhs7w202fxbrxd1c7fa-python3.7-click-7.0/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/nix/store/lma5qkl6r8vqrbhs7w202fxbrxd1c7fa-python3.7-click-7.0/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/nix/store/hnnz9g0kswahg630b4rwxh00l8bw2pmd-tmuxp-1.5.4/lib/python3.7/site-packages/tmuxp/cli.py", line 814, in command_load
    load_workspace(config[-1], **tmux_options)
  File "/nix/store/hnnz9g0kswahg630b4rwxh00l8bw2pmd-tmuxp-1.5.4/lib/python3.7/site-packages/tmuxp/cli.py", line 536, in load_workspace
    builder.build()  # load tmux session via workspace builder
  File "/nix/store/hnnz9g0kswahg630b4rwxh00l8bw2pmd-tmuxp-1.5.4/lib/python3.7/site-packages/tmuxp/workspacebuilder.py", line 184, in build
    for p, pconf in self.iter_create_panes(w, wconf):
  File "/nix/store/hnnz9g0kswahg630b4rwxh00l8bw2pmd-tmuxp-1.5.4/lib/python3.7/site-packages/tmuxp/workspacebuilder.py", line 315, in iter_create_panes
    w.select_layout(wconf['layout'])
  File "/nix/store/khrmcyr697xxwi1f616q7w8frskbw1sx-python3.7-libtmux-0.8.2/lib/python3.7/site-packages/libtmux/window.py", line 151, in select_layout
    raise exc.LibTmuxException(proc.stderr)
libtmux.exc.LibTmuxException: ['server exited unexpectedly']

3.json:

{
  "session_name": "0",
  "windows": [
    {      
      "options": {},
      "window_name": "bash",
      "layout": "f72d,271x66,0,0[271x33,0,0,31,271x32,0,34,35]",
      "panes": [
        {
          "shell_command": "bash",
          "focus": "true"
        },
        "nix"
      ],
      "start_directory": "/bakery2/oven1/personal/projects/hask2/bb-clean"
    }
  ]  
}

Sidenote: on the note of plugins, it might be interesting to create a CRIU plugin that can freeze certain processes, but it has a lot of limitations and I haven't figured out how to use it.

@deliciouslytyped Does it still happen with a minimal config?

{
  "session_name": "0",
  "windows": [
    {      
      "options": {},
      "window_name": "name",
      "panes": [
        "bash"
      ]
    }
  ]  
}

the window isn't staying open

Does it work without -S?

Edit: I see this is related to layout: https://github.com/tmux-python/tmuxp/issues/531#issue-542363502

I have a seemingly somewhat working codebase but the changes are a bit more extensive than I hoped, I'm going to be quite busy for a while so I might have to just dump a nasty diff / repo soon, so people can see it. I don't expect something to be done with it.

@deliciouslytyped yeah a rough diff in a branch would be enough to look closer. If it's done in a PR, we don't have to merge, but the advantage is it runs through our CI. you can also allow maintainers to edit in the PR

Sorry, this is really bad right now, but here it is, I just wanted to stop sitting on the code at least: archive.zip

There are some committed and uncommitted things, all on the main (checked out) branch.

@deliciouslytyped Thank you! Can you create a fork of this repo and commit it to a branch? It's just what I'm used to since it's GH, easier to diff / make sense of. No problem if the code is rough - I just want a clear presentation of what's happening more than anything :)

Is this still being worked on by any chance? I'm currently looking for an alternative to tmux_continuum&resurrect as it's largely just a bunch of bash scripts that make alot of "noise" with my various security utilities..

Still just the super messy stuff I have kludged up above - and I dont use it - it's not polished enough (not sure if it even works at all). I still have interest in this but haven't had a proper go at it since.

Are you interested in working on it?

I would be at present I don't have the time required to work on this it would be later this year before I could get around to being able to work on this.

Hi, I'm wary of adding more noise than contributing to the discussion, but this seems to be the right group of people to engage in this conversation with, so here goes.

I am a longtime user of resurrect (having abandoned continuum at some point a few years back for reasons I can't quite remember).

I am here because I have now decided that I'm a bit fed up with losing all the scrollback if I want to reboot. I'll list out my thoughts on the situation and what I would consider to be a step forward, keeping in mind the relative apparent difficulty of various hypothetical features. I'll also note I have very little exposure to tmuxp.

• 95% of the time or perhaps more, I'm good enough about monitoring battery state for reboots to be a premeditated event. I wouldn't really be trying to do any real work on a machine that is not configured well & running stable, or which has truly pitiful battery life. I am OK with having the process becoming a manual step.

• It seems that restoring shell history on a per pane basis is troublesome to implement, and I would argue that for practical purposes, this is not as important as restoring scrollback. If you just had to work out the exact history of a given pane, the restored scrollback of that pane should have you covered.

• I also never found much value in restoring running processes. Similarly to the above, having scrollback restored would make it really easy to see what used to be running in each window.

• This brings me to another point. If you are going after the holy grail, simply restoring the process running at the time, and restoring the command history into the shell, is actually not even enough. You should restore even more state, including but not limited to:

  • environment variables
  • defined shell functions
  • directory stack.

  What will you do if the thing running in a pane happened to be a shell function that was defined before? Restoring that ain't gonna be easy.

I think that's it. I don't want the holy grail, and I don't see anyone attempting a save/restore system for tmux that isn't some attempt at a holy grail, which proceeds well enough until it collapses under its own complexity.

All I need is a tool, whether it is integrated into tmux package manager or not, which has one command to do the following:

• persist window & pane layout for the session
• store current working directory per pane
• store entire pane scrollback per pane

...and a command to do the following:

• restore all that aforementioned state into the currently running session

If I had this I'd have a lot less anxiety around rebooting. Having a restore system that is as staggeringly complex as it would need to be to do the "holy grail" restoration has very high likelihood of being brittle, and that would itself contribute to the anxiety.

It does seem that maybe itd be better to have this discussion in the resurrect repo as it already implements the first two bullets well enough for my purposes.

Edit: It looks like my rant is a bit moot due to https://github.com/tmux-plugins/tmux-resurrect/pull/79 (AND RTFM https://github.com/tmux-plugins/tmux-resurrect/blob/master/docs/restoring_pane_contents.md), sorry guys.

I more or less agree with everything you said. The holy grail stuff was mostly me wondering out loud how good/bad it would work, and not too hard to try in isolation once surrounding tmuxp infrastructure exists (I think?). Indeed scrollback would get us a long way. Thank's for coherently stating a lot of my thoughts. And indeed, resurrect already covers a lot of this, but I had issues with it (can't remember either) and didn't want to start trying to complicate bash code further. :P A python implementation would also be a better base for further development to do whatever you want I think.