gpt4all icon indicating copy to clipboard operation
gpt4all copied to clipboard

csharp: update C# bindings to work with GGUF

Open cebtenzzre opened this issue 1 year ago • 3 comments

The C# bindings haven't been updated since the GGUF changes were merged. I had to remove the model type detection as it worked by checking the magic in the GGML header, but most models are handled by llama.cpp now so it's probably not important. If we want to add this back, it should go through gpt4all-backend API so it does not concern itself with the specifics of the file format.

The C# bindings now check llmodel_model_create2 for failure, and throw a useful exception instead of an access violation:

LLModel ERROR: Could not find any implementations for build variant: default
Unhandled exception. System.Exception: Model format not supported (no matching implementation found)
   at Gpt4All.Gpt4AllModelFactory.CreateModel(String modelPath) in /home/jared/src/forks/gpt4all/gpt4all-bindings/csharp/Gpt4All/Model/Gpt4AllModelFactory.cs:line 42
   at Gpt4All.Gpt4AllModelFactory.LoadModel(String modelPath) in /home/jared/src/forks/gpt4all/gpt4all-bindings/csharp/Gpt4All/Model/Gpt4AllModelFactory.cs:line 61
   at Program.<Main>$(String[] args) in /home/jared/src/forks/gpt4all/gpt4all-bindings/csharp/Gpt4All.Samples/Program.cs:line 13
   at Program.<Main>(String[] args)

This also fixes an issue with finding the implementation .dlls on Windows by making sure the path is absolute. Most users of Dlhandle use an absolute base path and were not affected, but the C# bindings do not call llmodel_set_implementation_search_path and use the default of ".", which is relative. On *nix, dlopen interprets paths relative to the current working directory, but on Windows they are relative to the search path (which may not include the current working directory).

Fixes #1534

cebtenzzre avatar Nov 14 '23 23:11 cebtenzzre

https://github.com/sdcondon/gpt4all/tree/csharp-fixes

jacoobes avatar Dec 24 '23 16:12 jacoobes

@sdcondon are you interested in making a pull request?

cebtenzzre avatar Dec 29 '23 21:12 cebtenzzre

@sdcondon are you interested in making a pull request?

Hi there. Sure thing, I'd be happy to to raise a PR (or PRs, if what's there is considered too much at once). There are a few things on my mind, though:

  • This one's been open for a while. It may be better to get it merged first? Especially since the base of my work is currently the head of this branch, not main. Is it still open because you're lacking people to shepherd C# stuff through? Or auto-validation shortfalls? I'm happy to put a bit of effort into that if you need it.
  • There are other active branches for the C# stuff. I found this one while looking at this, which includes a different approach to one particular thing (prompt context management). I humbly suggest that my way is better (see my comment on an issue), but I really don't want to tread on the toes of someone who's a regular contributor. @mvenditto
  • My changes obviously don't do anything to improve the somewhat dire state of the tests. Given that they're broken as it stands (some key ones try to use models that seem to have been removed), I'm not that uncomfortable with not sorting that out right now - but would be keen to take a look at that at some point afterwards.

So, any thoughts on the above would be gratefully received - but, first things first, the priority should probably be getting this one merged?

sdcondon avatar Dec 31 '23 12:12 sdcondon