abi
failed to generate code when uploading a screen recording
1、it will suceed when uploading a screenshot.
2、it will fail when uploading a screen recording.
the log from chrome console is as follows:
the log from backend service is as follows:
thanks for your help.
@brettyoung6678 how large is your screen recording? And which browser are you using?
That information will be helpful to me in debugging this issue.
browser : chrome version ：123.0.6312.123（64 bit）
screen recording: 12,545KB
i can not upload here because of the size limitation of github. i can provide a s3 link if you need.
thanks a lot, @abi
No need for the actual screen recording file. I just wanted to see how large the file was.
I'll check on my end if that should be okay.
One thing you could do is make the file smaller (just use some online compression site) and try it. I think it should work.
Same issue here. When generating screenshots, everything works fine, but whether it's screen recording or uploading local screen files, it fails to generate. The front-end indicates an error in generating the code.
I reviewed the backend logs and found that the system failed to process the video when uploading video files and attempting to generate code. Although the WebSocket connection was successfully established, and the server logs indicated that the code generation process had begun, an error occurred during the video file processing stage. The system tried to parse the video using the MoviePy library but encountered an IndexError with the message "list index out of range". The specific stack trace indicates that this error occurred while trying to read the video file's duration. Here is an excerpt from the error log:
Exception in ASGI application
OSError: MoviePy error: failed to read the duration of file C:\Users\mywinusername\AppData\Local\Temp\tmpfk6jogim.webm.
Here are the file infos returned by ffmpeg:
At the bottom of the error log, there is also an entry stating "C:\Users\mywinusername\AppData\Local\Temp\tmpfk6jogim.webm: Permission denied," indicating that access to this file was denied.
No need for the actual screen recording file. I just wanted to see how large the file was.
I'll check on my end if that should be okay.
One thing you could do is make the file smaller (just use some online compression site) and try it. I think it should work.
Yes, you are right. i can work when i compress the file into a smaller one. thanks a lot.
@brettyoung6678 Awesome, I'll try to make it work for larger files as well.
@hei2077 thanks for looking at all the logs. There's a problem browser recorded webm videos not including the duration which might be causing this problem. I have to fix this in s2c codebase but can you try recording the video using a different app or taking the recorded video and converting it to MP4. That might fix your problem. The problem could also be with the permissions of the temp directory. Could you try Googling and seeing if there's a fix for this. I'm using the Python tempfile lib: https://docs.python.org/3/library/tempfile.html to store the video before processing.
raise IOError(("MoviePy error: failed to read the duration of file %s.\n"
OSError: MoviePy error: failed to read the duration of file C:\Users\18759\AppData\Local\Temp\tmpg4nfqcmd.mp4.
Here are the file infos returned by ffmpeg:
@Herannn are you recording the video in the browser or separately outside the browser and then, dropping it in? The duration failure error should not happen with videos recorded outside the browser.
outside the browser. Below is the backend log.
INFO: Will watch for changes in these directories: ['E:\yingyong\screenshot-to-code\screenshot-to-code\backend'] INFO: Uvicorn running on http://127.0.0.1:7001 (Press CTRL+C to quit) INFO: Started reloader process [22312] using WatchFiles INFO: Started server process [44168] INFO: Waiting for application startup. INFO: Application startup complete. INFO: ('127.0.0.1', 52090) - "WebSocket /generate-code" [accepted] Incoming websocket connection... INFO: connection open Received params Generating vue_tailwind code for uploaded video using Llm.GPT_4O model... Using OpenAI API key from client-side settings dialog Using OpenAI Base URL from client-side settings dialog generating code... C:\Users\18759\AppData\Local\Temp\tmp8uf8q90o.mp4 ERROR: Exception in ASGI application Traceback (most recent call last): File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\moviepy\video\io\ffmpeg_reader.py", line 285, in ffmpeg_parse_infos line = [l for l in lines if keyword in l][index] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^ IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\uvicorn\protocols\websockets\websockets_impl.py", line 250, in run_asgi result = await self.app(self.scope, self.asgi_receive, self.asgi_send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in call return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\fastapi\applications.py", line 1054, in call await super().call(scope, receive, send) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette\applications.py", line 123, in call await self.middleware_stack(scope, receive, send) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette\middleware\errors.py", line 151, in call await self.app(scope, receive, send) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette\middleware\cors.py", line 77, in call await self.app(scope, receive, send) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette\middleware\exceptions.py", line 65, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette_exception_handler.py", line 64, in wrapped_app raise exc File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette\routing.py", line 756, in call await self.middleware_stack(scope, receive, send) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette\routing.py", line 776, in app await route.handle(scope, receive, send) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette\routing.py", line 373, in handle await self.app(scope, receive, send) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette\routing.py", line 96, in app await wrap_app_handling_exceptions(app, session)(scope, receive, send) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette_exception_handler.py", line 64, in wrapped_app raise exc File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\starlette\routing.py", line 94, in app await func(session) File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\fastapi\routing.py", line 348, in app await dependant.call(**values) File "E:\yingyong\screenshot-to-code\screenshot-to-code\backend\routes\generate_code.py", line 231, in stream_code prompt_messages = await assemble_claude_prompt_video(video_data_url) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\yingyong\screenshot-to-code\screenshot-to-code\backend\video\utils.py", line 21, in assemble_claude_prompt_video images = split_video_into_screenshots(video_data_url) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\yingyong\screenshot-to-code\screenshot-to-code\backend\video\utils.py", line 79, in split_video_into_screenshots clip = VideoFileClip(temp_video_file.name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\moviepy\video\io\VideoFileClip.py", line 88, in init self.reader = FFMPEG_VideoReader(filename, pix_fmt=pix_fmt, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\moviepy\video\io\ffmpeg_reader.py", line 35, in init infos = ffmpeg_parse_infos(filename, print_infos, check_duration, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\yingyong\ANACONDA\envs\poept312\Lib\site-packages\moviepy\video\io\ffmpeg_reader.py", line 289, in ffmpeg_parse_infos raise IOError(("MoviePy error: failed to read the duration of file %s.\n" OSError: MoviePy error: failed to read the duration of file C:\Users\18759\AppData\Local\Temp\tmp8uf8q90o.mp4. Here are the file infos returned by ffmpeg:
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 9.2.1 (GCC) 20200122 configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100 C:\Users\18759\AppData\Local\Temp\tmp8uf8q90o.mp4: Permission denied
