C# Bindings

[mvenditto]

Feature request

It would be nice to have C# bindings for gpt4all.

Motivation

Having the possibility to access gpt4all from C# will enable seamless integration with existing .NET project (I'm personally interested in experimenting with MS SemanticKernel). This could also expand the potential user base and fosters collaboration from the .NET community / users.

Your contribution

I have a working first version at my fork here.

Basically the library enables low-level access to the C llmodel lib and provides an higher level async API ontop of that.

If you think this could be of any interest I can file a PR.

NOTE: Where I live we had unprecedented floods this week and the power grid is still a bit unstable. If I go silent a couple days that's probably the reason.

[AndriyMulyar]

Yes, let's get C# bindings in! PR them to a subdirectory of the gpt4all-bindings directory!

[YongHuangSJTU]

Great to see this. For current C# API, can you expose LLModelPromptContext in GetPredictionAsync and other methods? LLModelPromptContext.PastNum is an important variable that will be modified inside C++, and it's important to use the value to know existing conversation token size to build a bot. Not sure if other values in LLModelPromptContext should be reused in a bot or not.

[mvenditto]

I'm working right now on providing an high-level API for chat message generation.

UPDATE

You can see here a basic chat sample with response regeneration (not upstream yet).

---

You can take a peek here csharp/feature/chat-api if you like. It could end up looking something like this:

using Gpt4All;
using Gpt4All.Chat;

var modelFactory = new Gpt4AllModelFactory();

using var model = modelFactory.LoadModel("./ggml-gpt4all-j-v1.3-groovy");

// create a new chat
IChatConversation chat = model.CreateNewChat();

// add an user message
chat.AddMessage(ChatRole.User, "Name 3 colors");

// ask for a response message from the "assistant"
var message = await model.GetStreamingMessageAsync(chat);

await foreach (var token in message.GetPredictionStreamingAsync())
{
    Console.Write(token);
}

At this point chat.Messages will contain:

1. ChatMessage{ AuthorRole=User, Content="Name 3 colors" }
2. ChatMessage{ AuthroRole=Assistant, Content="whatever the answer is"  }

And it will be possible to access the chat LLModelPromptContext to implement additional features. For example manipulating the LLModelPromptContext::PastNum to regenerate the response:

chat.Context.PastNum -= promptResponseTokens;
// ...
// generate another message

Will something like this provide enough flexibility ?

I'm trying to balance between a nice high-level but flexible enough API and exposing all the low level stuff as it is.

[YongHuangSJTU]

Thanks @mvenditto. Using GetPredictionStreamingAsync allows me to handle more low level things that Chat style API, so I need to access LLModelPromptContext there. In my local branch I simply change PredictRequestOptions from record to a class and update it inside each prompt, but I think it's a better idea to just remove PredictRequestOptions.cs and use LLModelPromptContext in all APIs instead?

[niansa]

It's been in for long now.

Please always feel free to open more issues as needed.