RuntimeError: for .NET framework 4.7.2.

[wonderingmark123]

NativeLibraryConfig.Default.Logging = true
NativeApi.llama_empty_call();

RuntimeError: The native library cannot be correctly loaded. It could be one of the following reasons: 
1. No LLamaSharp backend was installed. Please search LLamaSharp.Backend and install one of them. 
2. You are using a device with only CPU but installed cuda backend. Please install cpu backend instead. 
3. One of the dependency of the native library is missed. Please use `ldd` on linux, `dumpbin` on windows and `otool`to check if all the dependency of the native library is satisfied. Generally you could find the libraries under your output folder.
4. Try to compile llama.cpp yourself to generate a libllama library, then use `LLama.Native.NativeLibraryConfig.WithLibrary` to specify it at the very beginning of your code. For more informations about compilation, please refer to LLamaSharp repo on github.

My code is as simple as this

string modelPath = "C:\\Users\\Haotian\\AppData\\Local\\nomic.ai\\GPT4All\\orca-2-13b.Q4_0.gguf"; // change it to your own model path
                                                                                                  // Load a model
var parameters = new ModelParams(modelPath)
{
    ContextSize = 512,
};
using var model = LLamaWeights.LoadFromFile(parameters);
// Initialize a chat session
using var context = model.CreateContext(parameters);
var ex = new InteractiveExecutor(context);
ChatSession session = new ChatSession(ex);

This code works fine for my machine with .NET 8.0. However, I cannot run this code in the same machine for another project whose version is .NET framework 4.7.2.

I guess version is the problem. Could you help me fix this please?

[martindevans]

Could you try this again with the new 0.10.0 release?

If it's still a problem someone will need to dig further into why it's not loading with .NET Framework.

[wonderingmark123]

Thanks for your attention! There is still the same problem after updating to 0.10.0. And I tried both CPU and GPU backends.

[AsakusaRinne]

Could you please post the log when running it with NativeLibraryConfig.Instance.WithLogs() under .NET framework 4.7.2 here?

Furthermore, please try using NativeLibraryConfig.Instance.WithLibrary(<path>) to specify the path of native library file, which could be found in https://github.com/SciSharp/LLamaSharp/tree/master/LLama/runtimes/deps

[SINAPSA-IC]

Hello

Thank you! for the feedback! and for your Time!

For the kicks, I’ve created a WinForms App under .NET 4.7.2: while the Backend.CPU 0.10.0 gets added to the project, LlamaSharp does not benefit from this: Here’s the NuGet console:

Each package is licensed to you by its owner. NuGet is not responsible for, nor does it grant any licenses to, third-party packages. Some packages may include dependencies which are governed by additional licenses. Follow the package source (feed) URL to determine any dependencies.

Package Manager Console Host Version 6.9.1.3

Type 'get-help NuGet' to see all available NuGet commands.

PM> NuGet\Install-Package LLamaSharp.Backend.Cpu -Version 0.10.0

Attempting to gather dependency information for package 'LLamaSharp.Backend.Cpu.0.10.0' with respect to project 'WindowsFormsApp1', targeting '.NETFramework,Version=v4.7.2' Gathering dependency information took 576 ms Attempting to resolve dependencies for package 'LLamaSharp.Backend.Cpu.0.10.0' with DependencyBehavior 'Lowest' Resolving dependency information took 0 ms Resolving actions to install package 'LLamaSharp.Backend.Cpu.0.10.0' Resolved actions to install package 'LLamaSharp.Backend.Cpu.0.10.0' Retrieving package 'LLamaSharp.Backend.Cpu 0.10.0' from 'nuget.org'. Adding package 'LLamaSharp.Backend.Cpu.0.10.0' to folder 'H:\programe_vs\programe_vs_vs2022_desktop\WindowsFormsApp1\packages' Added package 'LLamaSharp.Backend.Cpu.0.10.0' to folder 'H:\programe_vs\programe_vs_vs2022_desktop\WindowsFormsApp1\packages' Added package 'LLamaSharp.Backend.Cpu.0.10.0' to 'packages.config' Successfully installed 'LLamaSharp.Backend.Cpu 0.10.0' to WindowsFormsApp1 Executing nuget actions took 762 ms Time Elapsed: 00:00:01.5834484

PM> dotnet add package LLamaSharp --version 0.10.0 Determining projects to restore... Writing H:__temp_folder_windows\tmpstyv5n.tmp info : X.509 certificate chain validation will use the default trust store selected by .NET for code signing. info : X.509 certificate chain validation will use the default trust store selected by .NET for timestamping. info : Adding PackageReference for package 'LLamaSharp' into project 'H:\programe_vs\programe_vs_vs2022_desktop\WindowsFormsApp1\WindowsFormsApp1.csproj'. error: Error while adding package 'LLamaSharp' to project 'H:\programe_vs\programe_vs_vs2022_desktop\WindowsFormsApp1\WindowsFormsApp1.csproj'. The project does not support adding package references through the add package command.

Usage: NuGet.CommandLine.XPlat.dll package add [options]

Options: -h|--help Show help information --force-english-output Forces the application to run using an invariant, English-based culture. --package Id of the package to be added. --version Version of the package to be added. -d|--dg-file Path to the dependency graph file to be used to restore preview and compatibility check. -p|--project Path to the project file. -f|--framework Frameworks for which the package reference should be added. -n|--no-restore Do not perform restore preview and compatibility check. The added package reference will be unconditional. -s|--source Specifies NuGet package sources to use during the restore. --package-directory Directory to restore packages in. --interactive Allow the command to block and require manual action for operations like authentication. --prerelease Allows prerelease packages to be installed.

PM>

Now, let’s try with a Console App (.NET Framework) under .NET 4.7.2:

The same error: (looks like a) the Console App must be just a Console App and that b) Apps under .NET 4.7.2 will not accept LlamaSharp 0.10.0, unlike .NET 8.0... So… cannot even compile these 4.7.2 Apps ☹ wtx…

PM> NuGet\Install-Package LLamaSharp.Backend.Cpu -Version 0.10.0

Attempting to gather dependency information for package 'LLamaSharp.Backend.Cpu.0.10.0' with respect to project 'ConsoleApp2', targeting '.NETFramework,Version=v4.7.2' Gathering dependency information took 5 ms Attempting to resolve dependencies for package 'LLamaSharp.Backend.Cpu.0.10.0' with DependencyBehavior 'Lowest' Resolving dependency information took 0 ms Resolving actions to install package 'LLamaSharp.Backend.Cpu.0.10.0' Resolved actions to install package 'LLamaSharp.Backend.Cpu.0.10.0' Retrieving package 'LLamaSharp.Backend.Cpu 0.10.0' from 'nuget.org'. Adding package 'LLamaSharp.Backend.Cpu.0.10.0' to folder 'H:\programe_vs\programe_vs_vs2022_desktop\ConsoleApp2\packages' Added package 'LLamaSharp.Backend.Cpu.0.10.0' to folder 'H:\programe_vs\programe_vs_vs2022_desktop\ConsoleApp2\packages' Added package 'LLamaSharp.Backend.Cpu.0.10.0' to 'packages.config' Successfully installed 'LLamaSharp.Backend.Cpu 0.10.0' to ConsoleApp2 Executing nuget actions took 374 ms Time Elapsed: 00:00:00.3869512 PM> dotnet add package LLamaSharp --version 0.10.0 Determining projects to restore... Writing H:__temp_folder_windows\tmpb30b03.tmp info : X.509 certificate chain validation will use the default trust store selected by .NET for code signing. info : X.509 certificate chain validation will use the default trust store selected by .NET for timestamping. info : Adding PackageReference for package 'LLamaSharp' into project 'H:\programe_vs\programe_vs_vs2022_desktop\ConsoleApp2\ConsoleApp2.csproj'. error: Error while adding package 'LLamaSharp' to project 'H:\programe_vs\programe_vs_vs2022_desktop\ConsoleApp2\ConsoleApp2.csproj'. The project does not support adding package references through the add package command.

Usage: NuGet.CommandLine.XPlat.dll package add [options]

Options: -h|--help Show help information --force-english-output Forces the application to run using an invariant, English-based culture. --package Id of the package to be added. --version Version of the package to be added. -d|--dg-file Path to the dependency graph file to be used to restore preview and compatibility check. -p|--project Path to the project file. -f|--framework Frameworks for which the package reference should be added. -n|--no-restore Do not perform restore preview and compatibility check. The added package reference will be unconditional. -s|--source Specifies NuGet package sources to use during the restore. --package-directory Directory to restore packages in. --interactive Allow the command to block and require manual action for operations like authentication. --prerelease Allows prerelease packages to be installed.

PM>

Sent from Mail for Windows

From: Rinne Sent: Wednesday, February 28, 2024 19:32 To: SciSharp/LLamaSharp Cc: SINAPSA-IC; Comment Subject: Re: [SciSharp/LLamaSharp] RuntimeError: for .NET framework 4.7.2.(Issue #508)

Could you please post the log when running it with NativeLibraryConfig.Instance.WithLogs() under .NET framework 4.7.2 here? Furthermore, please try using NativeLibraryConfig.Instance.WithLibrary() to specify the path of native library file, which could be found in https://github.com/SciSharp/LLamaSharp/tree/master/LLama/runtimes/deps — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

-- This email has been checked for viruses by Avast antivirus software. www.avast.com

[hikicoder]

Dunno if it helps, but it is possible to upgrade from .NET framework to .NET Standard or Core with the assistant from Microsoft itself... I've used it to move a Console App that opens up a bunch of WPF windows. Seems like it supports winforms as well.

https://dotnet.microsoft.com/en-us/platform/upgrade-assistant

[SINAPSA-IC]

Hello!

Thank you!

From what I know, in a Console App, inserting a using System.Windows.Forms and adding a Reference to them is enough to mix Console with Forms, like you said; I don't know what that tool from Microsoft does... pff.f now it's late hereabouts... will try and check tomorrow.

Yet, the problem with resolving the References of Abstractions in LLamaSharp.dll remains.

---

From: HikiWaltz @.> Sent: Wednesday, February 28, 2024 12:44 PM To: SciSharp/LLamaSharp @.> Cc: SINAPSA-IC @.>; Comment @.> Subject: Re: [SciSharp/LLamaSharp] RuntimeError: for .NET framework 4.7.2. (Issue #508)

Dunno if it helps, but it is possible to upgrade from .NET framework to .NET Standard or Core with the assistant from Microsoft itself... I've used it to move a Console App that opens up a bunch of WPF windows. Seems like it supports winforms as well.

https://dotnet.microsoft.com/en-us/platform/upgrade-assistant

— Reply to this email directly, view it on GitHubhttps://github.com/SciSharp/LLamaSharp/issues/508#issuecomment-1969887647, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BF3GLTRJLQGR2YEVPX7TAN3YV6JMDAVCNFSM6AAAAABDG74QBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRZHA4DONRUG4. You are receiving this because you commented.Message ID: @.***>

[AsakusaRinne]

Hi all, maybe I've found a way to resolve this problem.

install-package LLamaSharp

Using this command in package manager console could work with .NET Framework 4.7.2, at least it works for me! I'm looking forward to your feedbacks.

[SINAPSA-IC]

Hello

I have mentioned (recommended) you to the guys at Nomic AI - GPT4All, they are /in need of C# bindings for the program – mainly, a DLL that would intermediate communication with an LLM, starting with its loading.

I understand that the C# code that they were /left with is no longer useful, or adequate at all, for building such a DLL which would be able to also deal with Vulkan/kompute; in such a degree, that nobody can make it work anymore - much less build upon it. I, for one, am trying, this being the second day during which I encounter one problem and then another one and so on.

They’ve got a Discord channel just for discussing C# bindings – sorry, if this link doesn’t work – I’m new on Discord: https://discord.com/channels/1076964370942267462/1216123803403554838 (Nomic AI -> GPT4ALL -> C# bindings development)

Ofc, they have separate approaches for C# and Python, Windows and macOS, but I think your code could be integrated, even if only for loading an LLM.

You know best. I’m just saying, not keeping to myself.

Cheers, like 😊

From: Rinne Sent: Saturday, March 9, 2024 12:31 To: SciSharp/LLamaSharp Cc: Victor; Comment Subject: Re: [SciSharp/LLamaSharp] RuntimeError: for .NET framework 4.7.2.(Issue #508)

Hi all, maybe I've found a way to resolve this problem. install-package LLamaSharp Using this command in package manager console could work with .NET Framework 4.7.2, at least it works for me! I'm looking forward to your feedbacks. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

-- This email has been checked for viruses by Avast antivirus software. www.avast.com