sigviewer icon indicating copy to clipboard operation
sigviewer copied to clipboard

Published 20 hours ago • verified publisher icon cbrnr

A loophole

Open Yida-Lin opened this issue 8 years ago • 11 comments

@cbrnr Just noticed a loophole when debugging Stefan's file:

Before I thought string streams are always irregular streams, but turns out Stefan's file has a stream that is 5000Hz but is string only.

Could also be why that file doesn't load.

I am debugging more into this.

Sorry @stfnrpplngr it might take a while.

Yida-Lin avatar Mar 28 '17 03:03 Yida-Lin

@cbrnr @stfnrpplngr I actually successfully opened this file, though it took quite long loading the file because it has 170,000 events...

I have pushed new commits into SigViewer and Libxdf fixing the loopholes. (in a Pull Requested called fixEventTS, which hasn't been merged into master yet). Now the smaller file that Stefan sent earlier can be successfully opened. The 1.09GB one still can't due to allocation failure (the shortcoming of a 32bit build).

@cbrnr Clemens do you have anything to comment here?

2017-03-28

image

image

image

Yida-Lin avatar Mar 28 '17 04:03 Yida-Lin

Is this fix in #71?

cbrnr avatar Mar 28 '17 06:03 cbrnr

@cbrnr Hi Clemens, yes, together with the new commit in libxdf

Yida-Lin avatar Mar 28 '17 07:03 Yida-Lin

I still think that converting the original event stream from numeric to string isn't a great idea. However, maybe the easiest solution for us is to treat regular string streams with type sampledMarkers (or whatever the exact name is) as a special case. Specifically, we should remove all empty samples from such a stream and convert only non-empty strings to events.

cbrnr avatar Mar 28 '17 07:03 cbrnr

@cbrnr Hmm I am not sure I exactly understand. Are you saying if the string is empty we should ignore it?

Yida-Lin avatar Mar 28 '17 17:03 Yida-Lin

Yes, if the stream type is sampledMarker.

cbrnr avatar Mar 28 '17 17:03 cbrnr

@cbrnr wait, isn't the stream name or type arbitrarily entered in any way the user wants?

Also, if we filter those empty strings out, we won't see those events on the signals and it becomes an empty stream. Is that OK?

Yida-Lin avatar Mar 28 '17 17:03 Yida-Lin

Yes, you can choose the type but there are standards for meta data. I'm not sure if this type is in there yet.

This stream should contain some non-empty strings, otherwise what is the point of this signal? So if you discard all empty strings you should be left with the useful ones.

cbrnr avatar Mar 28 '17 17:03 cbrnr

@cbrnr seems the sampledMarker meta standard isn't there yet.

I see. I mean at least in Stefan's test file, it contains only empty strings, and if filtered out, there are no events left at all.

I can go ahead and make the changes, but might need another test file to better test the results.

Yida-Lin avatar Mar 28 '17 17:03 Yida-Lin

@stfnrpplngr is this intended? Shouldn't there be at least some non-empty strings in the marker stream?

cbrnr avatar Mar 28 '17 19:03 cbrnr

@stfnrpplngr @Yida-Lin any progress on this one?

cbrnr avatar Apr 25 '17 07:04 cbrnr