Possible Bug in pylsl.StreamInlet or StreamInfo

[Foucl]

I have spend days now trying to reliably reproduce this, without success. I think the following happens mostly when I (re)create multiple StreamOutlets during development.

Description

When I use pylsl.StreamOutlet in, the created streams often crashes the other end when trying to create the corresponding StreamInlet. The call to pylsl.StreamInlet(streams[0]) (where streams was obtained e.g. through pylsl.resolve_byprop('name', 'some_unique_name')) then results in the following traceback:

File "<ipython-input-6-666a607ba553>", line 14, in <module>
    inlet = pylsl.StreamInlet(streams[0])

File "c:\users\ddmitarbeiter\anaconda2\lib\site-packages\pylsl\pylsl.py", line 642, in __init__
    self.sample = self.sample_type()

TypeError: 'list' object is not callable

Further Debugging information

The streams[0] object I passed to in StreamInlet is definitely somehow corrupted: Both info.channel_count() and info.channel_format() return 0, most other methods just return an empty string. I tried to further debug the call to resolve_byprop or, respectively, the construction of the StreamInfo object. The handle passed from resolve_byprop to StreamInfo isn't None, so init returns immediately.

I am using pylsl from the ftp site and the liblsl dlls provided with that. I am on Windows, Anaconda Python 2.7, x64.

Reproduction

To reproduce this the first time, calling and cancelling the below code (LslTest / LslTest.send_triggers) a couple of times is sufficient. After one bad outlet has been created, most of the time, any method of creating an outlet with pylsl fails.

import time
import random
import pylsl

class LslTest(object):

    def __init__(self, stream_name='triggers'):
        self.stream_name = stream_name  
        print("created outlet with stream_name", self.stream_name)
        self.info = pylsl.StreamInfo(self.stream_name, 'Markers', 1, 0, 'string', 'myuidw43536')
        self.outlet = pylsl.StreamOutlet(self.info)

    def send_triggers(self):
        markernames = ['Test', 'Blah', 'Marker', 'XXX', 'Testtest', 'Test-1-2-3']
        begin = time.clock()
        #time.sleep(60)
        while True:
            self.outlet.push_sample([random.choice(markernames)])
            time.sleep(random.random()*3)
            if time.clock() - begin >= 20:
                return

o = LslTest('triggers15')

Btw: Is there a recommended way to 'destroy' unused StreamOutlets? I tried calling the __del__ method, which is the only option I found in the source and seems to work most of the time.

Edit: Clarified Reproduction

[cboulay]

As a first step can you try using the latest pylsl.py from this repository? The one on the FTP is slightly outdated.

It wasn't clear from your original post if the LslTest class you posted can produce the error initially, or if it can only produce the error after the initial error has been produced by Anaconda. Can you clarify?

[Foucl]

Thank you, I tried to clarify my original post. I am beginning to think it might have something to do with the university network? I will try at home. Currently, I am in the process of updating pylsl to this repo's master branch and the liblsl dlls to 1.12. I will report back.

[Foucl]

Updating

The update did not change the issue. Rather, creating a stream outlet in Python (by whatever means) now sometimes crashes python (or ipython) if the bug was 'triggered'.

Further Report

Waiting and rebooting (usually) helps. I still haven't found a reliable way of reproducing the bug. I am quite sure that it is not related to a particular PC. I think it is related to the number of outlets I create during a short period of time. Do you think it might help if I put the creation of the outlet (and the subsequent pushing of samples) in its own thread in my stimulus presentation script (right now, I do not use threading in it at all)? The stream I create is a simple one-channel string marker stream, without any additional stuff. The LabRecorder also reports the Stream as (invalid: bad lexical cast ...) in the GUI.

Different LAN

[will edit/update as soon as I have tried]

Please tell me if my issue is too bizarre / unlikely / idiosyncratic, I do not want to create unnecessary noise here.

[cboulay]

After one bad outlet has been created, most of the time, any method of creating an outlet with pylsl fails.

Is this true for an outlet with any parameters, or only parameters matching the previously created "corrupted" stream? i.e., if you change the name or the UUID does it still crash? I'm guessing there's a corrupt stream hanging around. @dmedine or @chkothe will know better than I, but I'm thinking that creating the outlet is not deleting the old stream and creating one anew, rather it is trying to reconnect and take control of the old stream, even if it's corrupt.

So maybe you create the corrupt stream when you try to reconnect to a stream that is in the process of being destroyed. Try decreasing the value passed to the max_buffered kwarg when creating the outlet and see if this prevents the old stream from lingering around so long.

Do you think it might help if I put the creation of the outlet (and the subsequent pushing of samples) in its own thread in my stimulus presentation script (right now, I do not use threading in it at all)?

This might help if whatever method you use to kill your Python process correctly cleans up child threads while not correctly cleaning up the main thread. If that happens, then maybe the stream controlled by the outlet owned by the child thread will be destroyed properly. However, this is not a good general solution.

If we can narrow down the problem further and determine that the stream is not being destroyed when it should be, then maybe we need to figure out a way to be more aggressive about destroying streams when Python crashes. An alternative solution might be to modify the C library itself so that when you create an outlet, liblsl first checks to see if an outlet already exists but is "corrupt" and if so then it destroys it (extra firepower?) and creates a new one.

[Foucl]

Is this true for an outlet with any parameters, or only parameters matching the previously created "corrupted" stream?

The latter. Changing name, id, channel count and/or channel dtype does not solve the issue. However, it only affects sockets created from python (or ipython). The Sender from LSL/liblsl/bin/ continues to work as expected.

The threading approach is working well thus far - no 'corruptions'. I am currently writing a test suite for my experiment. If I find a reliably reproduction method for this bug in the process, I will report it here.