Open-Assistant
Open-Assistant copied to clipboard
LAION-AI
Suggestion: ability to use a package manager (such as nix) to use additional software
The nix package manager let's you install software without admin permissions. This isn't a security flaw; it is designed to sandbox all software so that users installing software can't affect other users. For example, nix is how repl.it sandboxes users from each other. See this blog post.
As a generalization of #602 and related issues, I propose have an option to create a limited permission and limited resource "assistant" user that can install packages for it's self using nix (a new user would be created for each session, with the previous user deleted).
So for example, if the assistant needs to do math, it could use nix to install sympy (this happens nearly instantly and without much extra space if another user previously installed it; it uses copy on write) and then use sympy to do the math, obsoleting #602. If the user wants it to write a Haskell program or some other obscure language, it could use nix to install the language if it wants to test anything first.
This would be trained in the same way you would train it to use any other sort of external software. For example, we could explicitly include some examples where it uses sympy. We could also train it to prefer software that nix has locally cached (for efficiency reasons).
Oh this is a great idea. Let's see who wants to work on this issue and #602 :)
We need to be careful in using nix though as it is LGPL. We should only use it as a library and not modify their code.
how exactly do you imagine this to work? and why did you choose nix?
virtualization and sandboxing are widely established methods and countless approaches exist. instead of constraining to nix, why not just use docker, where the overlay FS layer can be deleted after each user? that way, no restriction to any particular software is necessary.
and in case we're mainly interested in python packages, why not just sandbox python? would be much more efficient than sandboxing users in an OS, like we'd have to do useradd, allocate home directories, etc. for each inference call.
Nix is far more efficient than docker for these type of use cases; this is part of why repl.it uses it.
Nix is first and foremost a package manager, and so unlike docker it automatically caches the installed software. Needing to install things from scratch each time is probably an unacceptable performance regression for most environments, unless we limit what the assistant is permitted to install (so they can be installed ahead of time).
It is also reproducible, which I think is very important in this context; we don't want the assistant to get bricked because of a software update for one of the tools it learned to use.
If we don't want to constantly make new OS users, we can use the nix-shell command, which creates a temporary environment with the requested packages installed. Or we can just reset the home directory and packages like you said. Actually, if we don't want super-user permissions, this is probably how we need to do it. Either nix-shell or manually reset the home directory.
In addition, afaik Docker by default needs super-user privileges to install things (I'm not super familiar with Docker). This is probably unacceptable from a security perspective in most environments.
As for how this work, I was thinking that instead of interacting directly with the user, in this mode the assistant interacts with a shell (such as a nix shell, or just a normal shell that can see nix). The assistant communicates with user using a assistant_tell and assistant_listen command. (I'll try to write up with an example when I'm back at a computer. But for example it would start with assistant_listen, interact with the shell like it was in a dialogue with it, and then use that information to write a assistant_tell command.) repl.it has the nice feature that if you try to use a command that is from a package you haven't installed, it automatically tries to install it. We could replicate that approach (although making this part reproducible might take a little extra effort).
Were you asking me @yk . I don't know which packages are good. I'm open to any. This was a discussion related to getting sympy to work to help with math data augmentation and maybe inference. Any sandbox you think is better is much appreciated.
I don't mean to use docker as a package manager or for the assistant to pull docker images, but to use docker as a sandboxing environment. nix may handle packages, but you'll still need to isolate the file system, you'll need to isolate the process groups, the network, etc. etc. docker does all that for you. just let the assistant run inside a docker container. if it wants to use nix, fine, but if it wants to use apt or pip or whatever, that's also fine, because at the end, I can just not commit the created overlay FS and the next user has a clean slate.
I didn't say nix was not a good package manager, just that there are a lot more ideal options to solve the sandboxing issue, as we need it.
if it wants to use nix, fine, but if it wants to use apt or pip or whatever
This sounds pretty slow tbh. Each time you reinstall nix itself, it would need to create a new package cache.
And I'm not saying we need to sandbox everything the assistant does; wouldn't just limiting permissions work? Like only giving it permission to home directory, not giving it access to any devices, etc... To be clear, I'm suggesting nix just so packages are sandboxed between different conversations and the users packages. I think the rest would just be normal permissions.
We could even lockdown internet access this way; we only allow it to access the internet through nix, and only to a pinned package list (which is check summed), see here. The check sum ensures it is only downloading data that already existed. I don't know a similar way this could work in docker (other than just pre-installing nix in it, which is redundant).
Even if we do use docker to sandbox the filesystem, I think nix is way better for efficiently installing packages (as long as we make sure all the docker images use the same installation cache).
That being said, I think just giving it raw access to a shell could be interesting too (if a bit slow). This would still let it install it's own software. I guess I saw #602 and my mind was already racing to generalization (and nix seemed like a pretty simple way to do that, since it also manages python packages).
(Oh, and nix also supports managing programming language package management in addition to application package management. Repl.it manages it's python packages with it instead of pip.)
This sounds pretty slow tbh. Each time you reinstall nix itself, it would need to create a new package cache.
I never said to re-install nix. it can be part of the base container. Also, caches for anything can easily be handled & mounted, that's never an issue. The main issue here is security & generality. What we'd like to do is to have the best tradeoff: the most secure while being the most general. Running a process in a docker container gives that process maximum freedom, i.e. large generality, while almost completely isolating it, i.e. large security. so it seems superior in both points.
In any case, we should care about this only if someone actually wants to take on either of these tasks.
In any case, I don't think the container affects data collection.
@ontocord @yk here is my suggested syntax for the assistant using software
> assistant_listen
Hi, can you find the square root of fifty for me? I want the exact symbolic value, not an approximation.
> assistant_tell "Sure, the exact value is "
> nix-shell -p python39Packages.sympy
> python -c "import sympy; print(sympy.simplify('sqrt(50)'))"
5*sqrt(2)
> assistant_tell "five times the square root of two. Is there anything else I can help with?"
> assistant_listen
Nope, thanks!
> assistant_tell "Okay, have a good rest of your day!"
What do you think? I could provide a couple examples using different software if you like it. This when then become a sort of software "prompting guide" or something.
I was also thinking about possibilities for inline syntax, if that is preferable. Something like {python -c "import sympy; print(sympy.simplify('sqrt(50)'))":5*sqrt(2)} where the assistant does {python -c "import sympy; print(sympy.simplify('sqrt(50)'))": and then the rest of the software fills in :5*sqrt(2)} (by actually activating the command).
@yk I could also collect some examples using different package managers if you like. For example, the second largest repo is the AUR (accessed using pacman) so I could include some examples using that. (In my opinion this is probably overkill though.)
Actually, I don't think we could use pacman since it's GPL, right (see https://github.com/LAION-AI/Open-Assistant/issues/603#issuecomment-1377714764)? If so, I guess there's no reason to collect data for it?
I could browse for repos with compatible licenses here. I think pip should be fine (MIT license) but I think the nix repos are nearly a perfect superset (the nix repos regularly grab from pip's repos).
Re syntax - I think any syntax is fine, as long as we can parse it. Galactica used the <work> tag, followed by python code in markdown format. they say they don't turn on the interpreter, but that's an easy matter. we need to figure out how to run the code per ^
Right, I guess as long as it can converted it doesn't matter. (I think I do agree with Yannick now that using a container is better than restricting permissions.)
Actually, I think there is even a way to automatically install things when a command isn't found, but it is slow and sometimes requires interactivity (like if you use python when it's not installed, it asks what version). It could make learning easier though (it would be shorter)?
Update: The dataset at https://www.kaggle.com/code/christopherdking/notebook0548e4d207/data?scriptVersionId=117563381 is python scripts that install their dependencies (including system dependencies) via nix. I can start working on applying this to the sympy data @Sobsz is working on (and I'm supposed to be helping with anyways XD). I wonder what other software we need, maybe other programming languages? 🤔 I could start trying to work on Google searches or similar as well.
@ontocord Should I start looking into executing at text generation time? Do you know what part of the codebase or who I would talk to do that?
I suppose we would also need to collect human data using the website, right? I wonder what the best UI for that is.
@ontocord Should I start looking into executing at text generation time? Do you know what part of the codebase or who I would talk to do that?
Yes. I have code base for this. It's an easy parse of the output. Ping me in DM.
This seems like it will be superseded by the plugins approach. If I'm missing something or this is still required feel free to reopen
