Server app and web UI?

[mscottgithub]

Issue you'd like to raise.

I may have misunderstood a basic intent or goal of the gpt4all project and am hoping the community can get my head on straight. I believed from all that I've read that I could install GPT4All on Ubuntu server with a LLM of choice and have that server function as a text-based AI that could then be connected to by remote clients via chat client or web interface for interaction. I was under the impression there is a web interface that is provided with the gpt4all installation.

I was under the impression this could all be done on private hardware and hosted privately, even choosing to not have it connected to the Internet or reliant on any cloud services. I set out to do that. I have a Dell R720 host with 2 CPUs, 192GB of RAM, ample amounts of SSD storage, and a GeForce RTX 3080 ti GPU card. I installed Ubuntu on it and am trying to get GPT4All functional on the server.

I have so far not been able to get it installed--but that is not necessarily the goal of this question. I want to know if this goal is achievable and, hopefully, with little effort. I just want to get it installed and functional and then be able to connect to the server from a desktop PC using a chat client or browser for interaction with the LLM of choice. Have I misinterpreted this project? Is this intended to be a client-only installed system for local interaction only? If I am correct and this is easily doable then I would appreciate any help I can get to find instructions on how to get it installed and functional. So far I have not been able to find any clear, step-by-step instructions on how to install it on Ubuntu to get it to even launch and load a model for interaction, much less have it host that model for use by client PCs.

Respectfully and hopefully,

Mike

Suggestion:

If I am wrong and this is not possible at the current state of development I would think this would not be a terribly challenging thing to accomplish. The hard work is getting the AI and language models functional--which this team has clearly done a great job of. Providing a web interface or chat client for remote clients, I would think, would not be near as challenging.

[dlippold]

As far as I can see GPT4All is a client-only application. In particular it has no web interface (there is other software which has a web interface). But it can be installed and used on an Ubuntu system (server) and can be used by remote clients (in particular with Windows or Linux) by using virtual remote desktops. Neither the clients nor the server require a connection to the internet and therefore they can be part of an intranet or of a private network.

To install GPT4All an a server without internet connection do the following:

1. Install it an a similar server with an internet connection, e.g. on a cloud server, as described on the projekt page (i.e. run the install script on Ubuntu).
2. Download all models you want to use later.
3. Create tar-files of the directory gpt4all, of the directory with the models and of the directory $HOME/.config/nomic.ai/, copy the tar-files to the server without an internet connection and extract the files on this server.

To use GPT4All from the clients:

1. Install a software for virtual remote desktops, e.g. a RDP service or a VNC service, on the server.
2. If necessary install the remote desktop client software on the clients (if you install a RDP service on the server it can be used by Windows clients without installing any software on the windows clients, because Windows already has the program Remote Desktop Connection).
3. Create an user account on the server for every user who wants to use GPT4All on the server and extract the tar-file of the directory $HOME/.config/nomic.ai/ in the user home directory. Maybe adapt the file $HOME/.config/nomic.ai/GPT4All.ini .
4. If the server with the virtual remote desktop service has an internet connection or if you want to increase the security of the service anyway configure the service to only accept connections form localhost or block the ports of the service by a firewall. Then an user must open an ssh tunnel (by using ssh -L) before the user can use GPT4All. I strongly advice to disable password authentication and use public key authentication for the ssh service (that is also possible for Windows clients).

So it is not a very easy solution but it works.

[JWhiteUX]

I have a similar need/desire.

I would like to run gpt4all on a homelab VM and make it available for other LAN devices via a web interface. The VM has an RTX 3060, 64GB of RAM, and 16 cores available.

VNC and RDP are non-starters as they present security issues and require clients to install applications.

Ideally users should be able to access the gpt4all instance by visiting the appropriate ip and port from any browser running on a lan client.

I've found limited support and documentation for this type of HomeLab setup. This would make using gpt4all available via mobile devices and other CPEs that would otherwise restrict the use of LLMs handling sensitive data without using client resources.

[colvdv]

This would be an extremely valuable feature to have!

I'd like to be able to remotely chat from my phone with my homelab. LAN access would easily enable me to do this from anywhere using NordVPN Meshnet to connect everything.