Add ML-KEM/ML-DSA support for C# wrapper

[tamasan238]

Description

Implementing ML-KEM and ML-DSA APIs in the C# wrapper. For this PR to work, you need to merge other PRs that you will add in comments.

Testing

Tested.

Checklist

• [x] added tests
• [x] updated/added doxygen
• [x] updated appropriate READMEs
• [ ] Updated manual and documentation

[devin-ai-integration[bot]]

🛟 Devin Lifeguard found 3 likely issues in this PR

• no-memory-leaks snippet snippet: Wrap the body of mlkem_test and mldsa_test in try/finally blocks (or use using-style helpers) so that MlKemFreeKey / DilithiumFreeKey are always called in the finally clause, ensuring the native keys are released even when an exception is thrown.
• check-all-return-codes snippet snippet: Capture the int result from wc_MlKemKey_Delete / wc_dilithium_delete, verify it is zero, and log or propagate an error if it is non-zero.
• free-allocated-memory snippet snippet: Move RandomFree(rng) (and similar frees) into a finally block or also invoke it inside the catch blocks so that every path after RandomNew() or wc_*_new() frees the allocation.

@tamasan238 please take a look at the above issues which Devin flagged. Devin will not fix these issues automatically.

[tamasan238]

Add _new/_delete API for ML-KEM/ML-DSA https://github.com/wolfSSL/wolfssl/pull/9039

Use case: Add C# examples using ML-KEM/ML-DSA https://github.com/wolfSSL/wolfssl-examples/pull/514

[JacobBarthelmeh]

Retest this please Jenkins:

Found unhandled org.jenkinsci.plugins.workflow.support.steps.AgentOfflineException 

Codespell warning

Error: ./wrapper/CSharp/README.md:106: Suppport ==> Support
Codespell found one or more problems

[gojimmypi]

Hi @tamasan238 -

Looks like some nice PQ updates for wolfSSL C#. Thank you!

I'm seeing some runtime errors:

Starting Cryptographic Tests...

Starting RNG test
RNG Test Passed

Starting ECC tests

[ ... snip other tests ... ]

Comparing Shared Secrets...
Curve25519 shared secret match.

Starting ML-KEM test

Starting ML_KEM_512 shared secret test ...
Testing ML-KEM Key Generation...
Generate Key Pair A...
MlKem key creation exception: System.EntryPointNotFoundException: Unable to find an entry point named 'wc_MlKemKey_New' in DLL 'wolfssl.dll'.
   at wolfSSL.CSharp.wolfcrypt.wc_MlKemKey_New(Int32 type, IntPtr heap, Int32 devId)
   at wolfSSL.CSharp.wolfcrypt.MlKemMakeKey(MlKemTypes type, IntPtr heap, Int32 devId) in C:\PR9040\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs:line 2876
An error occurred: Failed to generate key pair A.

C:\PR9040\wolfssl\wrapper\CSharp\Debug\x64\wolfCrypt-Test.exe (process 26580) exited with code -1 (0xffffffff).
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

I followed the instructions in the README update:

## PQC Suppport

To enable ML-KEM / ML-DSA API, add following options for `wolfssl` and `wolfCrypt-Test` projects:

HAVE_MLKEM
HAVE_MLDSA

As seen here:

It appears wc_MlKemKey_New is not in the wolfSSL codebase? Perhaps you meant wc_MlKemKey_Init or wc_MlKemKey_MakeKey ?

Unrelated Warnings

There are some warnings that seem to have snuck in over time, unrelated to this PR:

1>C:\PR9040\wolfssl\wolfssl\error-ssl.h(245,1): warning C5287: operands are different enum types 'wolfCrypt_ErrorCodes' and 'wolfSSL_ErrorCodes'; use an explicit cast to silence this warning
1>(compiling source file '../../src/crl.c')
1>dtls13.c
1>C:\PR9040\wolfssl\wolfssl\error-ssl.h(245,1): warning C5287: operands are different enum types 'wolfCrypt_ErrorCodes' and 'wolfSSL_ErrorCodes'; use an explicit cast to silence this warning
1>(compiling source file '../../src/dtls13.c')
1>dtls.c
1>C:\PR9040\wolfssl\wolfssl\error-ssl.h(245,1): warning C5287: operands are different enum types 'wolfCrypt_ErrorCodes' and 'wolfSSL_ErrorCodes'; use an explicit cast to silence this warning
1>(compiling source file '../../src/dtls.c')
1>internal.c
1>C:\PR9040\wolfssl\wolfssl\error-ssl.h(245,1): warning C5287: operands are different enum types 'wolfCrypt_ErrorCodes' and 'wolfSSL_ErrorCodes'; use an explicit cast to silence this warning
1>(compiling source file '../../src/internal.c')
1>wolfio.c
1>C:\PR9040\wolfssl\wolfssl\error-ssl.h(245,1): warning C5287: operands are different enum types 'wolfCrypt_ErrorCodes' and 'wolfSSL_ErrorCodes'; use an explicit cast to silence this warning
1>(compiling source file '../../src/wolfio.c')
1>keys.c
1>C:\PR9040\wolfssl\wolfssl\error-ssl.h(245,1): warning C5287: operands are different enum types 'wolfCrypt_ErrorCodes' and 'wolfSSL_ErrorCodes'; use an explicit cast to silence this warning
1>(compiling source file '../../src/keys.c')
1>ocsp.c
1>ssl.c
1>C:\PR9040\wolfssl\wolfssl\error-ssl.h(245,1): warning C5287: operands are different enum types 'wolfCrypt_ErrorCodes' and 'wolfSSL_ErrorCodes'; use an explicit cast to silence this warning
1>(compiling source file '../../src/ssl.c')
1>tls.c
1>C:\PR9040\wolfssl\wolfssl\error-ssl.h(245,1): warning C5287: operands are different enum types 'wolfCrypt_ErrorCodes' and 'wolfSSL_ErrorCodes'; use an explicit cast to silence this warning
1>(compiling source file '../../src/tls.c')
1>tls13.c
1>C:\PR9040\wolfssl\wolfssl\error-ssl.h(245,1): warning C5287: operands are different enum types 'wolfCrypt_ErrorCodes' and 'wolfSSL_ErrorCodes'; use an explicit cast to silence this warning
1>(compiling source file '../../src/tls13.c')
1>aes.c
1>arc4.c
1>asn.c
1>C:\PR9040\wolfssl\wolfssl\error-ssl.h(245,1): warning C5287: operands are different enum types 'wolfCrypt_ErrorCodes' and 'wolfSSL_ErrorCodes'; use an explicit cast to silence this warning
1>(compiling source file '../../wolfcrypt/src/asn.c')

I'm addressing those warnings in https://github.com/wolfSSL/wolfssl/pull/8946

[gojimmypi]

Also, I checked with anhu & confirmed additional macros need to be defined.

For instance, check out this section of the user_settings.h that I used for the ESP32. The ESP_WOLFSSL_ENABLE_MLKEM is a Kconfig setting, that ends up in a header as CONFIG_ESP_WOLFSSL_ENABLE_MLKEM:

#ifdef CONFIG_ESP_WOLFSSL_ENABLE_MLKEM
    /* Kyber typically needs a minimum 10K stack */
    #define WOLFSSL_HAVE_MLKEM
    #define WOLFSSL_WC_MLKEM
    #define WOLFSSL_SHAKE128
    #define WOLFSSL_SHAKE256

    /* Old code points to keep compatibility with Kyber Round 3. */
    /*   ./configure --enable-kyber=all --enable-experimental    */
    #if defined(CONFIG_WOLFSSL_ENABLE_KYBER)
        #define WOLFSSL_MLKEM_KYBER
        #define WOLFSSL_EXPERIMENTAL_SETTINGS
    #endif

    #if defined(CONFIG_IDF_TARGET_ESP8266)
        /* With limited RAM, we'll disable some of the Kyber sizes: */
        #define WOLFSSL_NO_KYBER1024
        #define WOLFSSL_NO_KYBER768
        #define WOLFSSL_NO_ML_KEM_1024
        #define WOLFSSL_NO_ML_KEM_768
        #define NO_SESSION_CACHE
    #else
        /* Only needed for older wolfssl versions, see mlkem.h */
        #define WOLFSSL_KYBER1024
        /* optional alternative sizes:    */
        /* #define WOLFSSL_KYBER768       */
        /* #define WOLFSSL_KYBER512       */
        /* -- or disable a specific one:  */
        /* #define WOLFSSL_NO_ML_KEM_1024 */
        /* #define WOLFSSL_NO_ML_KEM_768  */
        /* #define WOLFSSL_NO_ML_KEM_512  */
    #endif
#endif

[tamasan238]

Thank you for your review! I'll correct each item and contact again.

[tamasan238]

1️⃣ Fixed below.

1. typo

• s/Suppport/Support/
• s/encalsulate/encapsulate/

2. WinCE Error

• type, key, ... -> key, type, ...

3. configuration options

added some options for wolfSSL(wolfCrypt). // I think I wrote it before, but it seems that I blew it away when rebasing git commit. I'm sorry.

2️⃣ These APIs are currently being added in a separate PR. (https://github.com/wolfSSL/wolfssl/pull/9039)

• wolfCrypt/wc_mlkem.c:
  • wc_MlKemKey_New()
  • wc_MlKemKey_Delete()
• wolfCrypt/wc_dilithium.c:
  • wc_dilithium_new()
  • wc_dilithium_delete()

I would appreciate it if you could check it after referring to the above.

[gojimmypi]

Hi @tamasan238 - looks like you are making good progress on the C# library and examples!

I do have a question: why is the PQ code only conditionally compiled in? The current conditional code is primarily platform-based, no? (e.g. decorators specific to Windows CE). Do you think it would be better if the PQ code was always on, always available, with no need for manually defined macros?

[tamasan238]

Thank you for your review!

Yes, I think it is better to be available without any macros. But in my understanding, some errors may occur when a user builds wolfSSL without PQC macros.

If some PQC APIs are unavailable in wolfSSL, the C# wrapper test program(wolfCrypt-Test.cs) will output an error. So, I think that at least wolfCrypt-Test.cs requires ifdef (if in C#). In order to avoid confusing users, I thought it would be better to apply the same style in wolfCrypt.cs.

However, I believe it is also important to keep the code base and build procedures as simple as possible. I would like to hear your perspective.

[gojimmypi]

Hi @tamasan238

Yes, I agree that building the native C wolfSSL DLL would need attention. As the PQ code there is gated with macros defined in the user_settings.h, what do you think of updating that to always have the PQ code enable?

You could leave a brief comment around the C# declarations indicating which macros are assumed to have been enabled in the external wolfSSL DLL.

[tamasan238]

Oh, I see! That is the best plan. I'll do that. Thank you :)

[tamasan238]

I applied them. Is this style correct?

// Build test will fail now because PR https://github.com/wolfSSL/wolfssl/pull/9039 has not yet been merged.

[gojimmypi]

Hi @tamasan238

I fetched your latest code from this pr2 branch, and applied the patch from pr1.

I see there's a section added in the wrapper/CSharp/user_settings.h:

/* Enable ML-KEM, ML-DSA */
#define HAVE_MLKEM
#define WOLFSSL_WC_MLKEM
#define WOLFSSL_HAVE_MLKEM
#define WOLFSSL_DTLS_CH_FRAG
#define HAVE_DILITHIUM
#define WOLFSSL_WC_DILITHIUM
#define WOLFSSL_SHAKE128
#define WOLFSSL_SHAKE256

This only enables ML-KEM/ML-DSA support, eh?

The reason I ask, is the out-of-the-box experience is such that my C# client in this PR will not connect to my unmodified listener.

I see this error for the C# wolfSSL-TLS-Client example:

Windows - ca-cert.pem
Windows - dh2048.pem
Calling ctx Init from wolfSSL
Finished init of ctx .... now load in CA
SNI IS OFF
Ciphers : TLS13-AES128-GCM-SHA256:TLS13-AES256-GCM-SHA384:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384
Connected TCP
Connection made wolfSSL_connect
SSL version is TLSv1.3
SSL cipher suite is TLS_AES_256_GCM_SHA384
socket connection issue System.Net.Sockets.SocketException (0x80004005): An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.Socket.Send(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
   at wolfSSL.CSharp.wolfssl.wolfSSLCbIOSend(IntPtr ssl, IntPtr buf, Int32 sz, IntPtr ctx) in C:\PR9040\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL.cs:line 943
Error in write
freeing ssl handle
freeing ctx handle

C:\PR9040\wolfssl\wrapper\CSharp\Debug\x64\wolfSSL-TLS-Client.exe (process 16152) exited with code 0 (0x0).
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

Here's the message I see at the server (a stand-alone RPi with clean wolfssl):

waiting... listening on port 11111
Early Data was not sent.
SSL_accept error -345, peer did not return a certificate
wolfSSL error: SSL_accept failed
Loop iteration: 25

waiting... listening on port 11111

My server was build like this:

 ./autogen.sh
 ./configure --enable-all

and I'm running it like this ./server_loop.sh:

#!/bin/bash

count=1
THIS_PORT=11111

while true; do
    echo ""
    echo "waiting... listening on port $THIS_PORT"
    ./examples/server/server -v 4 -p "$THIS_PORT" -b 0.0.0.0
    ((count++))
    echo "Loop iteration: $count"
done

The failure occurs in wolfSSLCbIOSend, specifically this line 934:

if (con.Send(msg, 0, msg.Length, SocketFlags.None) == 0 && sz != 0)

Can you please confirm what you see when running the examples? Any special instructions needed?

[tamasan238]

This is strange behavior. I will check it right away.

[tamasan238]

I could reproduce that issue with unmodified wolfssl v5.8.2. So I think that problem is not related this PR (but it should be fixed by other PR...).

Communication between Server/Client under the C# wrapper directory should work fine. I have attached a video. Could you please try it out? https://youtu.be/h0vbJybUvNs * It needs wolfssl.com's google account

Examples of ML-KEM/ML-DSA usage can be found here. https://github.com/wolfSSL/wolfssl-examples/pull/514

[tamasan238]

jenkins retest this please

[gojimmypi]

Hi @tamasan238

I'm glad to see you were able to reproduce the connection problem that I encountered.

I don't recall seeing & resolving that issue, but it does not occur on my WIP https://github.com/wolfSSL/wolfssl/pull/8946

btw: Note your failing haproxy Test / v3.1.0 (pull_request) is resolved with https://github.com/wolfSSL/wolfssl/pull/9061

Perhaps if you refresh from upstream to fix HAProxy, it will also resolve the C# connection error? I didn't dig into the root cause.

I'll take a closer look at the ML-KEM/ML-DSA examples soon.

As for the code review: I see you followed the existing pattern in the wolfCrypt-tests. That's good. But I'm not sure I would have implemented that with all the throw new Exception's. The problem here is all the subsequent tests will not even be attempted as soon as the first test fails.

I would have implemented something differently: one that completed all the tests, showed results, then threw a single error at the end, or even just gracefully exit with a non-zero exit code. Refactoring that is well beyond the scope of the PR though, and I suspect someone had a specific objective in mind there. Perhaps you can find out why that pattern was chosen?

In any case: thank you for your work on the C# PQ. It will be a nice addition!

Cheers

[tamasan238]

Thank you for your confirmation!

Issue where wolfSSLCbIoSend() fails under specific conditions

I still haven't figured out the root cause of this issue.

Regarding throw new Exception

Indeed, even if a test fails, subsequent tests should still be executed. I also think it would be better to address this in a separate PR, so can we put this PR on hold for now? To be honest, I'm not really sure why it's implemented this way.

https://github.com/wolfSSL/wolfssl/pull/9039 should be merged soon. After confirming this, I will rerun the GitHub Actions/Jenkins tests and get back to you.

[tamasan238]

jenkins retest this please

[tamasan238]

The API addition PR(https://github.com/wolfSSL/wolfssl/pull/9039) has been merged! And by rebasing, I confirmed the following:

• The haproxy test issue now passes. (https://github.com/wolfSSL/wolfssl/pull/9061)
• Using the added API, wolfCrypt-Test.cs now succeeds.

Could you please re-check or/and merge this PR?

[gojimmypi]

I've confirmed the ML-KEM / ML-DSA test app runs successfully in VS 2022.

I didn't get a chance today to test client / server apps, nor a specific code review yet.

[ ... snip ... ]
Starting ML_KEM_1024 shared secret test ...
Testing ML-KEM Key Generation...
Generate Key Pair A...
Generate Key Pair B...
ML-KEM Key Generation test passed.
Testing ML-KEM Key Encode...
ML-KEM Key Encode test passed.
Testing ML-KEM Encapsulation...
ML-KEM Encapsulation test passed.
Testing ML-KEM Decode...
ML-KEM Decode test passed.
Testing ML-KEM Decapsulation...
ML-KEM Decapsulation test passed.
Comparing Shared Secrets...
ML-KEM shared secret match.

Starting ML-DSA test

Starting ML_DSA_44 shared secret test ...
Testing ML-DSA Key Generation...
ML-DSA Key Generation test passed.
Testing ML-DSA Key Export...
ML-DSA Key Export test passed.
Testing ML-DSA Key Import...
ML-DSA Key Import test passed.
Testing ML-DSA Signature Creation...
ML-DSA Signature Creation test passed. Signature Length: 2420
Testing ML-DSA Signature Verification...
ML-DSA Signature Verification test passed.
[ ... snip ... ]

[tamasan238]

jenkins retest this please

[tamasan238]

jenkins retest this please

[tamasan238]

@gojimmypi I fixed mistakes. Could you please check them?

[tamasan238]

Thank you for checking. There are still many things that need to be fixed. I'm learning a lot from you explaining your thinking in detail. I'll continue working on the fixes.

[tamasan238]

I've made several fixes and responded to some review comments. Please review the changes.

Note that the system drive on my development Windows machine has failed, so I'm unable to run tests locally using Visual Studio. As a result, there may be some basic errors present. I plan to fix these by referencing the test results from Jenkins/GitHub Actions. If you notice any issues, I'd appreciate it if you could let me know.

[tamasan238]

Additionally, I was advised via Slack to refer to the following regarding error values, but it appears they are not referenced in wolfCrypt.cs.

https://github.com/wolfSSL/wolfssl/blob/master/wolfssl/wolfcrypt/error-crypt.h

Please let me know what error values you consider desirable to set, or whether you think a reference to error-crypt.h should be added.

"D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp.sln" (default target) (1) ->
"D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL-TLS-PSK-Server\wolfSSL-TLS-PSK-Server.csproj" (default target) (6) ->
"D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj" (default target) (10:9) ->
(CoreCompile target) -> 
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(2960,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(3004,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(3047,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(3087,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(3132,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(3184,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(3327,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(3351,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(3380,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(3430,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(3482,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]
 D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfCrypt.cs(3531,24): error CS0103: The name 'BAD_FUNC_ARG' does not exist in the current context [D:\a\wolfssl\wolfssl\wolfssl\wrapper\CSharp\wolfSSL_CSharp\wolfSSL_CSharp.csproj]

[tamasan238]

Additionally, I note that the current code includes code to force the test to terminate prematurely, as suggested to me via Slack.

/* DEBUG CODE FOR REVIEW */
ret = BAD_FUNC_ARG;

[gojimmypi]

Additionally, I note that the current code includes code to force the test to terminate prematurely, as suggested to me via Slack.

/* DEBUG CODE FOR REVIEW */
ret = BAD_FUNC_ARG;

See the error codes for BAD_FUNC_ARG and others in wolfssl/wolfcrypt/error-crypt.h; it might be nice to have them all available in the C# wrapper, or you could just define the ones you use. Try to use consistent values if not importing the entire list.

You might consider something like this:

using System;
using System.Runtime.InteropServices;
using System.Security.Cryptography;
using System.Text;
using static wolfSSL.CSharp.WolfCryptErrorCodes; // << add this

namespace wolfSSL.CSharp
{
    public enum WolfCryptErrorCodes : int  // << add this
    {
        WC_FAILURE = -1,
                              // with all the other error codes, maybe even in a file of its own
        BAD_FUNC_ARG = -173
    }

    public class wolfcrypt
    {

        private const string wolfssl_dll = "wolfssl.dll";

and then either: return (int)BAD_FUNC_ARG; or (probably more invasive and problematic but arguably better) update the return value types like this:

        public static WolfCryptErrorCodes MlKemEncodePublicKey(IntPtr key, out byte[] publicKey)
        {

There's also the option of something simple for now and this PR: just defined the ones you use & leave a note:

/* TODO: add all error codes from wolfcrypt/error-crypt.h */
public const int BAD_FUNC_ARG = -173;

[gojimmypi]

Note that the system drive on my development Windows machine has failed, so I'm unable to run tests locally using Visual Studio. As a result, there may be some basic errors present.

Any luck on your Visual Studio? I'm seeing these compile-time errors:

Use of unassigned local variable 'privA'
Use of unassigned local variable 'pubA'
Use of unassigned local variable 'cipherText'
The name 'BAD_FUNC_ARG' does not exist in the current context
Use of unassigned local variable 'sharedSecretA'
Use of unassigned local variable 'sharedSecretB'
Use of unassigned local variable 'privateKey'
Use of unassigned local variable 'publicKey'
Use of unassigned local variable 'signature'

Consider:

private static void mlkem_test(wolfcrypt.MlKemTypes type)
        byte[] pubA = { }, privA = { }, cipherText = { }, sharedSecretA = { }, sharedSecretB= { };

and:

 private static void mldsa_test(wolfcrypt.MlDsaTypes type)
        byte[] privateKey = { };
        byte[] publicKey = { };
        byte[] signature = { };