[Feature Request] Edit Docker Container | Add Docker Compose ability

[Vinylwalk3r]

Okay, I think I have a rough idea here. First of disclaimer time I'm not UI designer at all. So I just throwing a rough and unpolished idea here. But lets hope it can inspire something great!

Add a way to edit a existing containers config (volumes, env variables, etc)

• So, for my first idea, in the "x" menu this could be an option when having a container selected: If chosen, a big text box could appear which allows all settings (from either it's compose code or run command) to be edited. See below for a rough example of such a text box

Extend "Run Image" functionality using Docker Compose / Run

• If you select an image and go to "x" -> "Run Image", instead of it just asking for a container name and then running, it could ask for a name, then ask if you want to input a compose code or run command:

• After the compose / run selection screen, a text box could appear that allows you to input your code / command: After this, it runs as usual.

Question of Isaiah and Compose support

I've been thinking, do you think Isaiah could handle compose codes? It would need to

1. "mkdir new-compose-project" then
2. "touch (or whatever command is necessary) docker-compose.yml "all the inputted compose code"
3. then "docker compose up" Would that be possible, in whatever implementation you choose?

Final Notes

Do you think any of this could be doable / fitting for Isaiah? Again, these images are just to help visualize my ideas. The final product (of implemented) would be much more polished.

[will-moss]

Hi again!

Thanks a lot for taking the time to open that issue and for providing all those details and design ideas (and no need to be a designer! Your mockups are already very useful).

I have been thinking about that feature and everything you've described, and here's what I think :

• For starters, I will definitely implement that new block. As you said before, I think this is the last missing piece to make Isaiah complete and able to cover 100% (or almost) of the needs of most people.

• Now, regarding the feature itself, I think I can foresee two places where it should appear :

  • When creating a container from an image : This is what you've described. One can either run an image without any option (as it is today), or define a configuration for the future container before running the image. To define a configuration, one should either copy-paste a docker-compose.yml file (ideally...with only one service, the service built from the currently-highlighted image), or copy-paste a docker run command. When using a docker-compose.yml file, Isaiah should do as you said (run touch docker-compose.yml, fill in that file, and run docker-compose up -d). When using a docker run command, Isaiah should just run the same command on the system (and potentially run checks to prevent mishappenings, like running "rm -rf /"...)

  • When managing "projects / services" : I think a new tab "Projects" or "Services" or "Stacks" should be added to Isaiah, like lazydocker does. The idea would be to :

    • Scan the disk / Query the Docker engine API to find all the docker-compose.yml files on the system.
    • For each docker-compose.yml found, create a new row on the UI in the browser in the "Projects" tab (on the left).
    • When one highlights a project row, they have three sub tabs in the inspector on the right : Logs (to see all the logs generated by all the resources inside the docker-compose.yml file), Services (to see all the resources that were instantiated or used by the docker-compose.yml file (images, networks, volumes, containers)), and Raw (to see the raw content of the docker-compose.yml file, and eventually, edit it directly from the interface, so as to edit the settings and restart the project when needed.
    • When one highlights a project row still, when they hit the "X" key to open the menu, they should be able to "up" the stack (run docker-compose up -d), "pull" the stack, "down" the stack, and "restart" the stack (down -> pull -> up).
    • Ultimately, from that tab "Projects" one shoud also be able to create a new project from scratch, by copy-pasting the content of a docker-compose.yml file in a big text box, like you described. In that scenario, Isaiah will run the commands you've listed (mkdir, touch, up)

• Regarding the "container editing" feature : I think that it could work that way (just paraphrasing your description really, for my own understanding and thiking-out-loud):

  • On the UI, the user highlights a container, and opens the menu. Then, they pick "edit container's config".
  • The user is now prompted for a docker run command / a docker-compose.yml file.
  • When the user confirms their input, isaiah shutdowns the container, and runs the user's input (either the docker run command, or a docker-compose up fed with the docker-compose.yml inputted.

All in all, unless you see limitations in my descriptions, I think I can start working on these features this week. The only "caveats / importants things not to take lightly" that I can see are :

• This can only work if Isaiah has access to the hosting system (a.k.a. deployed as a standalone app / multi-node) because it will certainly have to run bash commands directly (because the Docker Engine SDK / API isn't up to par with the CLI yet, afaik). So, an error should show when Isaiah is deployed via Docker, and a note should be added in the README.
• There should be checks in place to prevent users from running commands that could harm their machine / setup (prevent running anything other than docker run, disallow command chaining, ask for confirmation before running a command, check docker-compose.yml format, etc.).
• Ideally, having formatting helpers in the UI would be useful (line breaks for long docker run commands, syntax highlighting for docker-compose.yml files, visible tab characters for docker-compose.yml files, etc.).

Let me know if any idea / suggestion / limitation comes to your mind, and thanks a lot once again!

Looking forward to releasing that part !!

[Vinylwalk3r]

Glad they could be of help!

• About the container from image section. Yeah, having the choice to either specify a compose / run command or just start the image without any extra steps, that should be a thing.
• So in the "Run image options" menu, it should give four options:

1. Run as is
2. Docker Compose
3. Docker Run
4. Cancel

• If you choose compose or run, a comment could be added that says "Only supports one container. For multi-container stacks, please go to "Stacks / Projects" tab -> "Add new stack".

• I totally agree, having a dedicated "Stacks" tab would be superb. It could possibly be revealed when switching layout. Or to the right of the Container info section, as a new row?

• I really like your description of how the layouts and functions of the Stacks tab should be designed and work!

• If I may, maybe add a"Update" option in the X menu for a certain Stack t? I think that would be really usefull. So it "Downs" the stack, "pulls" new images and "Ups" the stack all in one go.

About the Container Edit feature

• Do you think Isaiah could make a docker compose / run from a container that it didnt create? I think I came across a command once that could check a containers config and recreate the command that made it. Super usefull in this case, I'll see if I can find it again!

Some other note:

• Do you think it's possible to create a popup in Isaiah, on first launch, that triggers if it detects it's run in a Docker environment? I'm proposing that it could be added to remind people that some features are not available when run as a container. Or a error message could be set to appear when they try to access a feature that is standalone only. Like the Sys shell or Stacks tab.
• The safety systems you describe would be necessary, I agree. Would Command chaining have to be disallowed though? It's super useful! (Im talking about the "/some command | next command" feature)
• Yeah, formatting help (could be as simple as having greyed out text that appears when the Docker Compose / Run text box is empty, with some sample code to show the formatting. Highlight syntax is such a great feature, please insert it if possible!

Good luck with coding it in and cant wait to try it out! Take your time, no hurry!

[will-moss]

Alright !

It's been a long time since our last conversation, and I'm glad to inform you that I have just published a new release.

As always, thank you a ton for helping, for providing details, and for asking questions.

I think this is the biggest release so far, and this is what it includes :

• Full support for "Docker stacks" (a.k.a. docker compose projects) :

  • You can now create stacks from a docker-compose.yml file
  • You can also up, down, restart, update, pause, and unpause any stack.
  • All the stacks are found on the local system using docker native commands, and presented to you.
  • You can, ultimately, edit an already-running stack (a.k.a. down, overwrite docker-compose.yml, and up)
  • As with other resources, you can view the live logs, services, and configuration of a stack.

• Support for updating a Docker container :

  • The container is stopped and removed.
  • The container's image is pulled
  • The container is recreated with its original settings

• Support for editing a Docker container :

  • You can now retrieve the original "docker run" command, and edit it inside your browser
  • After confirming, your container is stopped and removed, and your edited command is used to recreate the container.
  • If an error is met, the container is recreated automatically with the original "docker run" command before your edits.

• Extra environment variables to make everyone's life easier in the UI :

  • TABS_ENABLED : The list of tabs you want to display in the UI. By default, all are shown.
  • SORTBY_STACKS : The field used to sort stacks in the UI
  • STACKS_DIRECtORY : The directory used to store docker-compose.yml files generated by Isaiah.

• Additionaly, I've fixed a few bugs related to using the UI with the mouse.

• Additionaly, I've added (optional, enabled by default) syntax highlighting for docker-compose.yml files

Regarding your questions :

• I have added a blocking pop-up for all the features that require Isaiah to be running on the hosting system, and not in Docker. I thought, this should be less "invasive" than a pop-up appearing on launch, because there's already one for authentication, and then one for the overview. As a result, trying to use a "system-required" command will trigger a pop-up now. It's implemented.
• Two features are marked as experimental in the UI, because they're not based on Docker native capabilities. These features are Updating and Editing a container. Normally, there's nothing in Docker that natively enables these features. However, using third-party research, code, and snake oil, I was able to implement them. I would just like to point out that, there is a (small, I believe) risk of corrupting one's setup when using these features. Everything works well on my end, obviously, but I suspect more exotic environments to have issues.

Ultimately, regarding the "run image" feature, I have come to the conclusion that, given all the other features already implemented, adding an extra step to running an image may not be required or useful. I thought that, one could use the "Create stack" feature, or "Run image" then "Edit container" features. What do you think? Once again, I'm not opposed to it at all, I am just worried that I may repeat a lot of code when there are existing solutions.

Thank you very much again, and I hope this release works well for you!

PS : I still have to add a few security checks in a new version (to prevent someone from running anything other than "docker run" on their system).

Edit (19 august 2024) : A security check was added, along with documentation, and everything was released just now. After further testing, I'm confident that this feature is working, but I'll keep my eyes open until anyone confirms that it works for them as well.

[CptanPanic]

To go along with this, will it be possible in the future to have Isaiah create stacks, when it was started as a container?

[will-moss]

Thanks for joining this issue!

I will try to provide as many details as possible :

• Before I start, here's a useful link to understand the issue : https://github.com/docker/compose/issues/1033

• Quick summary : There is no official API in the Docker Engine to manage Docker Compose stacks.

• Because there is no official API, right now, what Isaiah does is create docker-compose.yml files, and run commands on the local system, like any user would do manually. (docker compose up / down / ps / restart / etc.

• For this reason, isaiah must have access to the hosting system (to create the docker-compose.yml files, but also to run the docker compose XXX commands).

• As a result, unless the Docker team implements, publishes, and documents a new Compose API inside the Docker Engine API, I'm afraid Isaiah won't ever be able to create stacks if it's inside a container. Because when it's inside a container, Isaiah only has access to the /var/run/docker.sock entrypoint for everything Docker.

Another solution would be to go with a 3rd party project that creates a Docker Compose API or translates a docker-compose.yml file into Docker Engine API calls, and integrate it into Isaiah, or as an external service to deploy and call from Isaiah. I'm sure this is feasible, but I'm worried about the maintenance : Since it wouldn't be an official project, what will happen when the official Docker project is updated? Is there a strong community around it? is there a strong documentation with known issues and edge cases? Are we sure that the 3rd party APi is robust and supports 100% of Docker Compose features? etc.

To be frank, I'm not opposed to working on the aforementioned solution, especially if it makes everyone's life easier. I would just like to be 100% confident that 1) the project is solid and well-maintained, and 2) the risk / benefit ratio is worth for people hosting isaiah (example: in case of data loss or service interruption due to Isaiah + 3rd party, I would be losing my mind).

If you have any idea or guidance on that issue, I'll gladly take it as I'm myself dissatisfied with the current error pop-up.

[CptanPanic]

@will-moss I know portainer doesn't use regular docker-compose is this how they do it?

[will-moss]

I have tried to do my research (reading Portainer's source code and other projects) and I think that Portainer has implemented their own yml-to-API engine. They make a distinction between Portainer Stacks, and Docker Compose Stacks. When using native Docker Compose Stacks, indeed, nothing can be done (up, down, edit, pull, etc.), it's all disabled. However, when using Portainer stacks, everything's working again.

Here's a screenshot : https://ibb.co/F4rGP8y And another one : https://ibb.co/McC7Lv3

They explicitly mention : "This stack was created outside of Portainer. Control over this stack is limited.". Therefore I believe that, when using native Docker Compose Stacks, Portainer has the same limitations as Isaiah.

I have tried looking into their dependencies, source code, and even asking ChatGPT, and it really seems to me that they have implemented their own "Compose Layer" to handle it themselves. And searching for "Compose" in their issues reveals that, indeed, they have a few native features that are unsupported or causing bugs.

Being the only developer (so far) on the project, I think I don't have what it takes to undertake this development (add support for Docker Compose like Portainer did), unless I can reuse code from Portainer. Once again, I'm not against it, but I would feel more confident working on it as a team or using the robust development of a previous team before me.

Note : I have looked into libcompose, and it's deprecated. There's also compose-go, but it doesn't seem to support anything other than parsing files... Any other idea is always appreciated 🙏

[will-moss]

It's been a little more than a month since that feature was implemented, and everything seems to be working as intended.

Unless anyone raises a new issue or any improvement to work on, I'll close that issue for now.

[ForceConstant]

@will-moss I missed this added feature, but tried it and it didn't work. So maybe I am misunderstanding. So I started isaiah using 'docker run...' and am unable to create a stack. Is this not what is meant?

[will-moss]

Hi,

No worries. Maybe I should update the README on this feature.

Here's the full breakdown :

• Back before July 20th (when this issue was opened), Isaiah could not manage Docker Compose Projects (stacks) at all. I mean : list, up, down, update a project started with docker-compose was completely...out and not implemented at all.

• On August 16th, I released the feature. From that day onwards, Isaiah is able to discover the Docker Compose Projects on the system, and manage everything (up, down, restart, update, edit docker-compose.yml from the browser, etc.)

• On August 28th, @CptanPanic mentioned that Isaiah wasn't able to manage stacks if it was deployed via Docker. And that is, indeed a current limitation (not only a limitation of Isaiah itself, but of all the programs that manage Docker, including Portainer to a certain extent). In order to manage Docker stacks, an access to the system is required, because natively, Docker Compose is only accessible via CLI, not via the API exposed via the Docker socket (/var/run/docker.sock). So, if Isaiah is in a Docker Container, without access to the system, it can't manage stacks. (This is also partially true for Portainer, but I figured that the Portainer developers had implemented their own "docker-compose.yml-to-API" layer to get around the need for docker-compose CLI).

• The current issue was opened to add support for "Docker Compose ability", and if I'm not mistaken, we fully have it now. Shall we want to manage stacks like Portainer, even when deployed via Docker, I think it should pertain to a new dedicated issue. My previous reply (link) is relevant here.

Let me know if I can provide more information! For now, I would generally advise to run Isaiah on the system directly, using the install script.