Occasional hang when closing device

[vidstige]

Intermittently say one in ten when closing a device that has had its two streams started, it will hang when calling Device.Close(). I have previously closed the two streams.

I quickly looked at the code and it looks as if the streams are closed again when the device is closed. Perhaps this is something that OpenNI does not like?

[falahati]

Hi Vidstige,

What will happen if you ignore closing of streams? Let device close the streams.

[vidstige]

Sorry I checked my code more closely and I was actually not closing the streams. I'm just calling Stop() on them and then setting the reference to null. Perhaps this sometimes causes a finalizer to be run or something like that? It might also be a bug in the OpenNI. Can you please check through the code and see if you can spot anything. I will look through the OpenNI2 2.2.0.33 code and see if I can find anything suspect.

Do you have device? Can you reproduce? Otherwise I can try to write a small reproducer.

[falahati]

It would be great if you could write a piece of code to reproduce the problem. Unfortunately I am little busy these days and I cant write a reproducer code soon.

[vidstige]

I could not attach a zip file with the reproducer, but here is the code in essence.It does not always hangs although I force the garbage collection and wait for pending finalizers. Never do this except when trying to reproduce issues like we're doing here. Note that it does not happens all the time, but only say one in ten. This leads me to think its an issue with OpenNI. Can you reproduce?

public partial class MainWindow : Window
{
    private Device _device;        
    private VideoStream _depthStream;
    private VideoStream _colorStream;

    public MainWindow()
    {
        InitializeComponent();
    }

    private void Window_Loaded(object sender, RoutedEventArgs e)
    {
        OpenNI.Initialize();
        _device = Device.Open(null);

        _depthStream = _device.CreateVideoStream(Device.SensorType.DEPTH);
        _colorStream = _device.CreateVideoStream(Device.SensorType.COLOR);
        _depthStream.onNewFrame += onNewFrame;
        _colorStream.onNewFrame += onNewFrame;
        _depthStream.Start();
        _colorStream.Start();
    }

    private void onNewFrame(VideoStream vStream)
    {
        Console.WriteLine("frame");
        vStream.readFrame().Release();
    }

    private void Window_Closed(object sender, EventArgs e)
    {
        _depthStream.Stop();

        _colorStream.Stop();
        _colorStream = null;
        _depthStream = null;
        GC.Collect(1000, GCCollectionMode.Forced, true);
        GC.WaitForPendingFinalizers();
        _device.Close(); // Hangs here

        OpenNI.Shutdown();
    }
}

[falahati]

Sorry, I almost forgot this issue. I will take a look at this and get back to you.

[vidstige]

That would be great! I'm suspecting its either the c layer of the wrapper or a bug inside OpenNI itself. I will try to look into the sources myself if I have time.

[falahati]

Fixed, I hope.

[vidstige]

Where you able to reproduce the bug and place a break point? Why do you otherwise think this fixes the problem?

[falahati]

I was able to reproduce the bug.

I modified your code a little and here is the reproduction code:

class Program
{
    private static Device device;
    private static VideoStream depthStream;
    private static VideoStream colorStream;

    static void Main(string[] args)
    {
        Random rndRandom = new Random();
        int i = 0;
        while (true)
        {
            i++;
            OpenNI.Initialize();
            device = Device.Open(null);
            Console.WriteLine(i + " -> Start");

            depthStream = device.CreateVideoStream(Device.SensorType.Depth);
            colorStream = device.CreateVideoStream(Device.SensorType.Color);
            depthStream.OnNewFrame += OnNewFrame;
            colorStream.OnNewFrame += OnNewFrame;
            depthStream.Start();
            colorStream.Start();

            System.Threading.Thread.Sleep(rndRandom.Next(1000, 3000));

            depthStream.Stop();
            colorStream.Stop();
            colorStream = null;
            depthStream = null;
            GC.Collect(1000, GCCollectionMode.Forced);
            GC.WaitForPendingFinalizers();
            Console.WriteLine(i + " -> Stopping");
            device.Close(); // Hangs here

            OpenNI.Shutdown();
            Console.WriteLine(i + " -> Stopped");
        }
    }

    private static void OnNewFrame(VideoStream vStream)
    {
        Console.WriteLine(vStream.SensorInfo.GetSensorType() + " -> New Frame");
        VideoFrameRef vframe = vStream.ReadFrame();
        System.Threading.Thread.Sleep(300);
        vframe.Release();
    }
}

Also it is not possible to put a break point there. Code hangs in managed to un-managed transition. There is no specific code in our wrapper or OpenNI it-self. This usually mean there is a problem with marshaling. I already knew that this problem could only happen when there is a conflict between OpenNI's event thread and main thread.

Best way to avoid this problem was simply stop raising events when stream is not active. To prevent user from using ReadFrame method to create a dead lock.

[vidstige]

Wow, great job. Having insight in the threading model makes it a lot easier I guess.

[NeilHarbin0]

I am using vb.net winforms, and I run in to this problem even with the latest available .net wrapper. I have found that the chances of it happening are greatly increased if I add a winforms timer being enabled and disabled. I don't understand why the timer has an effect, but this should help to reproduce the problem more easily.

My forms vb code: ImageCaptureForm.txt

I did also try it by modifying the reproduction code @falahati posted in C# with modifications to remove the color stream, and locks instead of thread sleeps, but it happens with that as well, although it takes many minutes.

class Program
{
	private static Device device;
	private static VideoStream depthStream;
	private static Object lockObject = new Object();

	static void Main(string[] args)
	{
		Random rndRandom = new Random();
		int i = 0;
		OpenNI.Initialize();
		while (true)
		{
			i++;
			device = Device.Open(null);
			Console.WriteLine(i + " -> Start");
			depthStream = device.CreateVideoStream(Device.SensorType.Depth);
			depthStream.OnNewFrame += OnNewFrame;
			depthStream.Start();
			System.Threading.Thread.Sleep(rndRandom.Next(100, 300));
			lock (lockObject)
			{
				depthStream.Stop();
				depthStream = null;
				GC.Collect(1000, GCCollectionMode.Forced);
				GC.WaitForPendingFinalizers();
				Console.Clear();
				Console.WriteLine(i + " -> Stopping");
				device.Close(); // Hangs here
				Console.WriteLine(i + " -> Stopped");
			}
		}
	}

	private static void OnNewFrame(VideoStream vStream)
	{
		lock (lockObject)
		{
			Console.WriteLine(vStream.SensorInfo.GetSensorType() + " -> New Frame");
			VideoFrameRef vframe = vStream.ReadFrame();
			vframe.Release();
		}
	}
}

[falahati]

I will take a look into this. Can you try an older version to see if the problem is there as well or just newly introduced again?

[NeilHarbin0]

Yes, the problem exists in the older version 2.2.0.33 as well as the current version.