gobot

gobot copied to clipboard

Hey, I've been trying to get the tello_opencv.go example working.

The issue is the ffmpeg.StdinPipe() doesn't seem to be a valid pipe.

When the code tries to do ffmpegIn.Write(pkt), it logs the error write |1: broken pipe, causing io.ReadFull(ffmpegOut, buf) to reach EOF.

I'm running MacOS 10.13.4.

The problem might be with the ffmpeg command not properly recognizing that input will be coming from pipe 0 (stdin).
Do different OS need different args formatting for ffmpeg input/output?

Hi @emf42 that is a very interesting question. According to the docs for ffmpeg that is the default pipe in, however I think for sure worth exploring.

@deadprogram, ya I've been trying all sorts of args formatting but still get the same error. Did you have to do anything special for yours to work?

Just the args in the code itself here: https://github.com/hybridgroup/gobot/blob/master/examples/tello_opencv.go#L37-L38

One thing perhaps worth trying would be running that command just on your command line and seeing what happens, such as some error.

ffmpeg -i pipe:0 -pix_fmt bgr24 -vcodec rawvideo -an -sn -s 960x720 -f rawvideo pipe:1

$ ffmpeg -i pipe:0 -pix_fmt bgr24 -vcodec rawvideo -an -sn -s 960x720 -f rawvideo pipe:1
ffmpeg version 4.0 Copyright (c) 2000-2018 the FFmpeg developers
  built with Apple LLVM version 9.1.0 (clang-902.0.39.1)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.0 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
  libavutil      56. 14.100 / 56. 14.100
  libavcodec     58. 18.100 / 58. 18.100
  libavformat    58. 12.100 / 58. 12.100
  libavdevice    58.  3.100 / 58.  3.100
  libavfilter     7. 16.100 /  7. 16.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  1.100 /  5.  1.100
  libswresample   3.  1.100 /  3.  1.100
  libpostproc    55.  1.100 / 55.  1.100

Theres the output. Execution does not halt.

How about this: check to see if you are even getting video data back from the Tello, by adding a fmt.Println("here") right before this line: https://github.com/hybridgroup/gobot/blob/master/examples/tello_opencv.go#L75

Perhaps you are not even getting video frame data from drone.

I am getting the video data. The pipe closed error stems from the ffmpegIn.Write(pkt) in that same block.

@deadprogram I just tried it a few more times, and now I am getting this error:

tello_opencv[2836:86136] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'nextEventMatchingMask should only be called from the Main Thread!'

No idea on that, sorry. Obviously some internal macOS error, but ?

It looks to be an openCV issue, not macOS

tello_opencv[3236:110730] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'nextEventMatchingMask should only be called from the Main Thread!'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff401af32b __exceptionPreprocess + 171
	1   libobjc.A.dylib                     0x00007fff67829c76 objc_exception_throw + 48
	2   AppKit                              0x00007fff3de5b297 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 4167
	3   libopencv_highgui.3.4.dylib         0x0000000005016a2c cvWaitKey + 351
	4   libopencv_highgui.3.4.dylib         0x0000000005012149 _ZN2cv9waitKeyExEi + 34
	5   libopencv_highgui.3.4.dylib         0x00000000050121af _ZN2cv7waitKeyEi + 34
	6   tello_opencv                        0x00000000041194d3 Window_WaitKey + 19
	7   tello_opencv                        0x000000000410915c _cgo_4c3040cb4e6c_Cfunc_Window_WaitKey + 28
	8   tello_opencv                        0x0000000004054f90 runtime.asmcgocall + 112
)
libc++abi.dylib: terminating with uncaught exception of type NSException
SIGABRT: abort
PC=0x7fff68593b6e m=12 sigcode=0

Generally the main thread and highgui windows are handled by https://github.com/hybridgroup/gocv/blob/master/highgui.go#L34

Since that problem just started, you might want to see if some lingering process is running?

Heres something interesting. If I remove the go func and take everything out of the work function, there are no problems writing the video data to ffmpegIn. https://pastebin.com/XTD2WE1R

When I try to add back the go function for reading from ffmpegout, I get the same error as above, but theres no pipe errors.

Do you get the message "Connected" in your log?

Yes I do.

Have you got any other ideas on how to create an OpenCV image mat from the video data? I havent been able to find anything else to work.

Hi @deadprogram, desperate to get this working. Have you got any ideas on a possible workaround, like feeding from mplayer to opencv instead of using ffmpeg? Or possibly restructuring the code so execution occurs in the main function instead of a gobot work function - that solved the write error but brings the NSInternalInconsistencyException when trying to create an image mat from stdout data.

You might trying adding this to the program:

func init() {
    runtime.LockOSThread()
}

It might help in this case.

For sure, I will look into this in the next couple of days, but I have a few deadlines coming up first.

Unfortunately that made no difference.

I think the issue is something to do with the goroutine. Writing to stdin works fine until the goroutine starts. The broken pipe error only begins once the goroutine attempts to read from stdout. Possibly something to do with synchronization across stdin and stdout?

I would expect is more a matter of OpenCV, threads, and macOS. If you search the internets, you will find some references to this. Do not think at all related to os.Exec and friends.

Not to do with os.Exec. To do with the goroutine and synchronization of variables across the multiple threads.

Hi @emf42 please take a try with the latest Gobot, GoCV, OpenCV. Also, make sure you are using the latest Gobot examples. I have tested the latest changes on macOS and they are now working for me.

@deadprogram on macos high sierra 10.13.6, gobot 1.12, opencv 3.4.2, and gocv 0.16; same issue trying every solution mentioned here