tator
tator copied to clipboard
Optimize latencies for server-side media util
Tator provides an innovative server-side media manipulation service with GetFrame
and GetLocalizationGraphic
-- A common use of these utilities is to fetch multiple frames or multiple related RoIs.
GetFrame supports sending back a tiled image of multiple frames. This can be used to lower latency of fetching nearby related frames in some UI applications. GetLocalizationGraphic could benefit from a similar trick. Because the extraction occurs in a child ffmpeg process, there is limited need for additional middleware for a scatter/gather to be implemented. If the Schema was extended to support multiple localizations in GetLocalizationGraphic
; it could fetch N localization via asynchronous ffmpeg processes, join on them, and then create a tile image to return in one REST call. Some profiling should be done to set a limit on the number of N in this concept.