ReadReport() memory leak

[kaczart]

Great library Mike, thanks a lot for this huge work.

The problem is when I want to fill InputReport with data:

InputReport = device.ReadReport();

where InputReport is:

HidLibrary.HidReport InputReport = new HidLibrary.HidReport(device.Capabilities.InputReportByteLength);

It's a very large of leak memory, specially when I read reports in async mode from device (for eg. 100 times for a second). In few minutes it can takes hundreds of megabytes.

I'm beginner in c# programming and didn't find solution for problem desrcibed above ;(

greetings!

[mikeobrien]

Kaczart,

Unfortunately that code is pretty old and I haven't had the time to give it any TLC. At this point I just have it out there as a reference. Your best bet is to fork/improve.

m

On Sun, Aug 14, 2011 at 10:37 AM, kaczart < [email protected]>wrote:

Great library Mike, thanks a lot for this huge work.

The problem is when I want to fill InputReport with data:

InputReport = device.ReadReport();

where InputReport is:

HidLibrary.HidReport InputReport = new HidLibrary.HidReport(ukp.Capabilities.InputReportByteLength);

It's a very large of leak memory, specially when I read reports in async mode from device (for eg. 100 times for a second). In few minutes it can takes hundreds of megabytes.

I'm beginner in c# programming and didn't find solution for problem desrcibed above ;(

greetings!

Reply to this email directly or view it on GitHub: https://github.com/mikeobrien/HidLibrary/issues/11

[jrockfl]

Kaczart,

Did you find a solution? I just noticed the same thing. Great library Mike, thank you so much! I'm working with an Endicia usb scale and this is pretty fun.

[kaczart]

jrockfl,

unfortunately I had no time and no skill enough to fix this issue :(

greetings

[Miraculix]

The problem ist that the HidReport and Devicedata is not freed, so it waits for a garbagecollection. It's an easy fix though to release it after it has been processed...

[kaczart]

Finally I had to fix this issue and I share what I found

1. First of all, every single Read/Write methods in library check if device is connected what costs a lot of performance because IsConnected method calls each time EnumerateHidDevices(). In practice, when I send big package of data from the device, PC Host is slower than HID device and some packets are lost due to limited ring buffer (32reports by default, can be extended to 512 but it's not final solution). ps. you can extend your ring buffer from 32 to max 512 reports calling: NativeMethods .HidD_SetNumInputBuffers((int )device.ReadHandle, 512); //It will work for XP and later.
2. Secondly, in EnumerateHidDevices() is HUGE memory leak due to "yield return devicePath;" in line 69 and NativeMethods.SetupDiDestroyDeviceInfoList(deviceInfoSet); will never be called.

My solution:

1. Do not call IsConnected every Read/Write. Make additional isConnected variable in HidDevice: public bool isConnected; Set variable to true after successful connect (e.g. in your Connect() function): device.isConnected = true; Update it in every DeviceEventMonitorInserted() and DeviceEventMonitorRemoved() events, eg: if (Inserted != null ) Inserted(); replace with: if (Inserted != null ) { isConnected = true; Inserted(); }
   and: if (Removed != null ) Removed(); replace with: if (Removed != null ) { isConnected = false ; Removed(); } Than check this variable every Read/Write call function: if (isConnected) instead of if (IsConnected) Every time you need to check if device is connected, check state of public variable isConnected. It works for me - 0-1% processor usage instead of 10-20% for read thread.

2. in HidDevice.cs line #69 delete: //yield return devicePath; It will allow to NativeMethods .SetupDiDestroyDeviceInfoList(deviceInfoSet); be called propetly. Next, after NativeMethods .SetupDiDestroyDeviceInfoList(deviceInfoSet) add: foreach (string devicePath in devices) yield return devicePath; It will fix HUGE memory leak (about 256MB for each 1MB received data in my case)

Hope it heps.

[Martin-Schlodinski]

awesome, thanks for your hints, kaczart!!!

[macaba]

Here is my suggested workaround for the performance issue that kaczart (and I) noticed: https://github.com/macaba/HidLibrary/commit/8f1864e1351ccca4772d5bbc18a0f94d7ba07fe7

Usage is along the lines of:

device.ConnectionCheckOverride = device.IsConnected;
methodRequiringFasterUSBReadWrite();
device.ConnectionCheckOverride = false;

This reduced my bulk USB transfer test case execution time from 159 seconds to 47 seconds.

I wasn't keen on kaczart's solution simply because the IsConnected property is used in HidDeviceEventMonitor, so needed to retain the original functionality of enumerating currently connected USB devices. Simply put; I don't think the Inserted and Removed events will ever be called in kaczart's code (from a brief look through it).

[exupero]

Checking for a device connection on every read and write is a major performance bottleneck in high-frequency applications. In my particular case it was 30x faster to assume the device was connected rather than re-checking for a device connection; round trips dropped from about 70ms to 2ms.

[JoshWobbles]

So I just wrote a couple extension methods to handle my own IO for rapid pooling usage:

        [DllImport("hid.dll", SetLastError = true)]
        static internal extern bool HidD_SetOutputReport(IntPtr hidDeviceObject, byte[] lpReportBuffer, int reportBufferLength);

        [DllImport("kernel32.dll", SetLastError = true)]
        static internal extern bool ReadFile(IntPtr hFile, [Out] byte[] lpBuffer, uint nNumberOfBytesToRead, out uint lpNumberOfBytesRead, IntPtr lpOverlapped);
        public static bool FastWrite(this HidLibrary.HidDevice device, byte[] outputBuffer)
        {
            try
            {
                if (NativeMethods.HidD_SetOutputReport(device.Handle, outputBuffer, outputBuffer.Length))
                    return true;
                else
                    return false;
            }
            catch
            {
                return false;
            }
        }
        public static ReadStatus FastRead(this HidLibrary.HidDevice device, byte[] inputBuffer)
        {
            try
            {
                uint bytesRead;
                if (NativeMethods.ReadFile(device.Handle, inputBuffer, (uint)inputBuffer.Length, out bytesRead, IntPtr.Zero))
                {
                    return ReadStatus.Success;
                }
                else
                {
                    return ReadStatus.NoDataRead;
                }
            }
            catch (Exception)
            {
                return ReadStatus.ReadError;
            }
        }