Typescript and C# Bindings no longer work after #779

[benobytes]

System Info

Apologizes if this is already currently being addressed but I have noticed I am unable to use the C# bindings after the change done in #779. To my understanding the bindings are no longer working due to the fact that these native functions don't exist: File : gpt4all\gpt4all-bindings\csharp\Gpt4All\Bindings\NativeMethods.cs

    [DllImport("libllmodel", CallingConvention = CallingConvention.Cdecl, ExactSpelling = true)]
    [return: NativeTypeName("llmodel_model")]
    public static extern IntPtr llmodel_gptj_create();

    [DllImport("libllmodel", CallingConvention = CallingConvention.Cdecl, ExactSpelling = true)]
    public static extern void llmodel_gptj_destroy([NativeTypeName("llmodel_model")] IntPtr gptj);

    [DllImport("libllmodel", CallingConvention = CallingConvention.Cdecl, ExactSpelling = true)]
    [return: NativeTypeName("llmodel_model")]
    public static extern IntPtr llmodel_mpt_create();

    [DllImport("libllmodel", CallingConvention = CallingConvention.Cdecl, ExactSpelling = true)]
    public static extern void llmodel_mpt_destroy([NativeTypeName("llmodel_model")] IntPtr mpt);

    [DllImport("libllmodel", CallingConvention = CallingConvention.Cdecl, ExactSpelling = true)]
    [return: NativeTypeName("llmodel_model")]
    public static extern IntPtr llmodel_llama_create();

    [DllImport("libllmodel", CallingConvention = CallingConvention.Cdecl, ExactSpelling = true)]
    public static extern void llmodel_llama_destroy([NativeTypeName("llmodel_model")] IntPtr llama);

It looks like these functions were removed in #779 in the gpt4all-backend/llmodel.h file. It also looks like a few of these removed functions are also referenced in the typescript bindings. File: gpt4all\gpt4all-bindings\typescript\index.cc

llmodel_model create_model_set_type(const char* c_weights_path) 
  {

    uint32_t magic;
    llmodel_model model;
    FILE *f = fopen(c_weights_path, "rb");
    fread(&magic, sizeof(magic), 1, f);

    if (magic == 0x67676d6c) {
        model = llmodel_gptj_create();  
        type = "gptj";
    }
    else if (magic == 0x67676a74) {
        model = llmodel_llama_create(); 
        type = "llama";
    }
    else if (magic == 0x67676d6d) {
        model = llmodel_mpt_create();   
        type = "mpt";
    }
    else  {fprintf(stderr, "Invalid model file\n");}
    fclose(f);
    
    return model;
  }

Information

• [ ] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [X] backend
• [X] bindings
• [ ] python-bindings
• [ ] chat-ui
• [ ] models
• [ ] circleci
• [ ] docker
• [ ] api

Reproduction

Any attempt to use the C# bindings leads to

Unhandled exception. System.AggregateException: One or more errors occurred. (Unable to find an entry point named 'llmodel_mpt_create' in DLL 'libllmodel'.)

Expected behavior

Perhaps the bindings just needs an additional wrapper, I am not familiar with the code yet but I will attempt to come up with a solution.

I am new to github, so if there is anything I missed or maybe am wrong go easy on me! :).

[AndriyMulyar]

Thanks for the report, we are aware and fixing all bindings currently!

[mvenditto]

C# bindings will be fixed when #763 is merged

[mudler]

just for reference: similarly golang bindings are broken as well

[mudler]

@mvenditto can you share what you did on C# to fix the bindings? that's what I'm seeing in golang bindings right now:

8:56AM DBG [gpt4all] Attempting to load
8:56AM DBG Loading model gpt4all from ggml-gpt4all-j
8:56AM DBG Loading model in memory from file: /home/mudler/_git/LocalAI/models/ggml-gpt4all-j
load_gpt4all_model: error 'Success'

seems that this snippet:

    auto model = llmodel_model_create2(fname, "auto", &new_error);

returns a nil model, and populates the error with a "Success" message. any idea what's going on? @AndriyMulyar @niansa

[mudler]

fix for golang bindings in https://github.com/nomic-ai/gpt4all/pull/981

[niansa]

This seems to be fixed.

Please always feel free to open more issues if you have anything else.