gpt4all
gpt4all copied to clipboard
csharp: update C# bindings to work with GGUF
The C# bindings haven't been updated since the GGUF changes were merged. I had to remove the model type detection as it worked by checking the magic in the GGML header, but most models are handled by llama.cpp now so it's probably not important. If we want to add this back, it should go through gpt4all-backend API so it does not concern itself with the specifics of the file format.
The C# bindings now check llmodel_model_create2 for failure, and throw a useful exception instead of an access violation:
LLModel ERROR: Could not find any implementations for build variant: default
Unhandled exception. System.Exception: Model format not supported (no matching implementation found)
at Gpt4All.Gpt4AllModelFactory.CreateModel(String modelPath) in /home/jared/src/forks/gpt4all/gpt4all-bindings/csharp/Gpt4All/Model/Gpt4AllModelFactory.cs:line 42
at Gpt4All.Gpt4AllModelFactory.LoadModel(String modelPath) in /home/jared/src/forks/gpt4all/gpt4all-bindings/csharp/Gpt4All/Model/Gpt4AllModelFactory.cs:line 61
at Program.<Main>$(String[] args) in /home/jared/src/forks/gpt4all/gpt4all-bindings/csharp/Gpt4All.Samples/Program.cs:line 13
at Program.<Main>(String[] args)
This also fixes an issue with finding the implementation .dlls on Windows by making sure the path is absolute. Most users of Dlhandle use an absolute base path and were not affected, but the C# bindings do not call llmodel_set_implementation_search_path and use the default of "."
, which is relative. On *nix, dlopen interprets paths relative to the current working directory, but on Windows they are relative to the search path (which may not include the current working directory).
Fixes #1534
https://github.com/sdcondon/gpt4all/tree/csharp-fixes
@sdcondon are you interested in making a pull request?
@sdcondon are you interested in making a pull request?
Hi there. Sure thing, I'd be happy to to raise a PR (or PRs, if what's there is considered too much at once). There are a few things on my mind, though:
- This one's been open for a while. It may be better to get it merged first? Especially since the base of my work is currently the head of this branch, not main. Is it still open because you're lacking people to shepherd C# stuff through? Or auto-validation shortfalls? I'm happy to put a bit of effort into that if you need it.
- There are other active branches for the C# stuff. I found this one while looking at this, which includes a different approach to one particular thing (prompt context management). I humbly suggest that my way is better (see my comment on an issue), but I really don't want to tread on the toes of someone who's a regular contributor. @mvenditto
- My changes obviously don't do anything to improve the somewhat dire state of the tests. Given that they're broken as it stands (some key ones try to use models that seem to have been removed), I'm not that uncomfortable with not sorting that out right now - but would be keen to take a look at that at some point afterwards.
So, any thoughts on the above would be gratefully received - but, first things first, the priority should probably be getting this one merged?