AegisubDC icon indicating copy to clipboard operation
AegisubDC copied to clipboard

Add automation function to get frame

Open seproDev opened this issue 2 years ago • 5 comments

Adds new functionality to the Lua automation API that exposes a video frame. Here is a lua code snippet showing how this currently works:

frame = aegisub.get_frame(current_frame) -- current_frame is integer
frameWithSubtitles = aegisub.get_frame(current_frame, true) -- subtitles are baked in to the video if needed

width = frame:width() -- The colon is important here as frame:width()  = frame.width(frame) 
print("Frame width: " .. width) -- Frame width: 1920
height = frame:height()
print("Frame height: " .. height) -- Frame height: 1080

pixelString = frame:getPixelFormatted(x, y)
print("Pixel String: " .. pixelString) -- Pixel String: &H0073FF&

pixelInt = frame:getPixel(x, y)
print("Pixel Raw: #" .. string.format("%x", pixelInt)) -- Pixel Raw: #ff7300

In general having people test this before merging would be great.

seproDev avatar May 06 '22 23:05 seproDev

It should be possible to use lua_automation_version to set the version as 4.1. This way those that want to check the function can check against the version string.

To enable future proofing, keep the feature in aegisub instead of aegisubdc.

Ristellise avatar May 07 '22 07:05 Ristellise

Ah actually in the past functions have been added without increasing the version, so I guess it's fine. https://github.com/Aegisub/Aegisub/commit/d660f7f2b023a71e90df27f1c51ba009a2d30301 If someone needs to test for a specific feature this is better anyway:

if not aegisub.get_frame then
    return
end

seproDev avatar May 07 '22 08:05 seproDev

Ah actually in the past functions have been added without increasing the version, so I guess it's fine. Aegisub/Aegisub@d660f7f If someone needs to test for a specific feature this is better anyway:

if (!aegisub.get_frame) then
    return
end

Guess that will work...

Ristellise avatar May 07 '22 12:05 Ristellise

Okay I dropped the direct array access and added an option to retrieve frames with subtitles. The last thing remaining is checking if any of the logic needs to be adjusted to account for flipped and pitch. The avisynth provider sets flipped to true here: https://github.com/Ristellise/AegisubDC/blob/d4e6c9afef17953c2d62b874665a1bfb62949b32/src/video_provider_avs.cpp#L317 So potentially all avisynth videos are flipped? Would be great if someone with a working avisynth setup could try this out.

Edit: Looking at the code I think the flip is a flip along the x-axis, while the pitch is the total number of bytes in a line not pixels. I'll adjust the code to account for that, though I can't test if it actually works as I am not using avisynth.

seproDev avatar May 07 '22 14:05 seproDev

For future reference: I tested this with AviSynth (and other) videos now, and all the pixel positions seem to come out correctly.

arch1t3cht avatar Aug 07 '22 23:08 arch1t3cht