Locked dll on windows

When using the library, the dlls librdkafka and zlib are locked by the w3p process and can't be deleted/updated until the webservice is stopped. In a webapi hosted in iis on windows server 2008 R2/ windows 10, this prevent the option to replace bin folder to update the service on the fly

I wonder if it's possible to avoid this lock?

The dotnet client should call FreeLibrary() when it is done with librdkafka: https://msdn.microsoft.com/en-us/library/windows/desktop/ms683152(v=vs.85).aspx

Seems like the first answer here may provide a solution: http://serverfault.com/questions/503721/replacing-dll-files-while-the-application-is-running scripting that could make it painless.

I guess it may be possible to do the same automatically on app startup - copy the dll to a temp filename in a different directory and then load that. But is there a temporary place on the fs that's guaranteed to exist and be writable on every system? or could we create a ram disk somehow? and if so is this bad in any way? All of this sounds a bit hacky to me, and with quite a bit of possibility for things to go wrong I haven't thought about.

it would possibly be good to call FreeLibrary(), but i think we can put that at low priority given how most apps will use the library. it'd be non-trivial to do automatically - we'd need to track globally how many instances of Producer / Consumer exist to implement this in their Dispose methods, and then there is the Library class, which potentially calls into librdkafka without even any producer or consumer instances existing.

going to mark this as won't fix. if someone comes up with a great solution though, we will.

For anyone facing the same issue, I fixed it by putting the dlls somewhere else on the server (in my case a unmanagedLib folder in a specific place of the server) and loading the dlls in my app before any call to confluent.kafka (code below) On deployment, i suppressed the x64/x86 folder copied in bin folder because Library constructor would call LoadLibrary on those dll (on net451 moniker)

An alternative way should be to just put x64 versions of librdkafka / zlib and other in System32 (and x86 version in SysWOW64) where they will be loaded automatically by DllImport (of course it must be updated whenever we change the librdkafka version)

This is more a hack to resolve our deployment tool (which we can't change for the moment)

[DllImport("kernel32.dll", EntryPoint = "LoadLibrary", SetLastError = true)]
public static extern IntPtr LoadLibrary(string dllToLoad);

[DllImport("kernel32.dll", EntryPoint = "FreeLibrary", SetLastError = true)]
public static extern bool FreeLibrary(IntPtr hModule);

public static bool LoadUnmanagedLib()
{
	string platform = IntPtr.Size == 8 ? "x64" : "x86";
	var unmanagedDirectory = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "unmanagedLib",  platform);
	zlibHandle = LoadLibrary(Path.Combine(unmanagedDirectory, "zlib.dll"));
	if (zlibHandle == IntPtr.Zero)
	{
		throw new Win32Exception(Marshal.GetLastWin32Error(), "failed to load zlib.dll in " + unmanagedDirectory);
	}
	librdkafkaHandle = LoadLibrary(Path.Combine(unmanagedDirectory, "librdkafka.dll"));
	if (zlibHandle == IntPtr.Zero)
	{
		throw new Win32Exception(Marshal.GetLastWin32Error(), "failed to load librdkafka.dll in " + unmanagedDirectory);
	}

	return zlibHandle != IntPtr.Zero && librdkafkaHandle != IntPtr.Zero;
}


public static void FreeUnmanagedLib()
{
	if(librdkafkaHandle != IntPtr.Zero)
	{
		FreeLibrary(librdkafkaHandle);
	}

	if (zlibHandle != IntPtr.Zero)
	{
		FreeLibrary(zlibHandle);
	}
}

i guess we could implement an Library.Unload method.

@mhowlett , or it would help to get the pointer to the handle so clients can unload it. I have resolved this issue before by @treziac explanation here , but now with the latest library updates I'm running into the same locking issue.

@treziac workaround you suggested is not working with the new library updates. I manually load the libraries from a shared folder before i create a new instance of producer or consumer, for some reason manual load is not recognized and I get an error "Error Loading librdKafka.dll" from the bin/Win7/x64

However I did resolve the loading issue with Library.Load() method, parameter to the load method is absolute path to Librdkafka.dll

Whenever I recycle app pool or deploy a new build. I get this error

We are seeing this same issue as well; I haven't yet found an acceptable workaround with v0.11.5

i meet the same issue, i do not want to stop iis to deploy every time.

Just ran into this problem. I solved this by manually keeping track of consumers / producers and disposing them on AppDomain.AppDomainUnload before unloading the native library via kernel32.dll's FreeLibrary. I wonder, if this can't be done in the library similarly: e.g. register all IClients in Library and register an EventHandler for AppDomain.AppDomainUnload which disposes all IClients before unloading the native library. This shouldn't even concern .NET core apps, as AppDomain.AppDomainUnload won't fire to the best of my knowledge.

We also have this problem. We used tfs for continuous deployment, but with the introduction of Kafka in our project, this has become a real hell. Is there really no solution after 4 years?

What i suggest is can manual point kafka dll from other location instead from bin folder

//Manual Point librdkafka.dll location var kafkapath = kafkaSettings.GetValue("LibrdkafkaPath", ""); if (!Library.IsLoaded && !string.IsNullOrWhiteSpace(kafkapath)) { Library.Load(kafkapath); Log.Information($"Load dll {kafkapath}"); }