dive icon indicating copy to clipboard operation
dive copied to clipboard

[FEATURE] Trans-code videos and run-pipelines in one stage (or just same time) instead of two

Open mattdawkins opened this issue 4 years ago • 9 comments

Need the ability to launch both transcode and (test) pipelines at the same time for overnight / background runs

mattdawkins avatar Jan 13 '21 01:01 mattdawkins

Due to #602 this will be impossible.

subdavis avatar Mar 02 '21 15:03 subdavis

This doesn't mean they have to actually occur at the same time, just occur at the same time from the user's perspective via 1 button click (they can still occur sequentially in the actual backend)

mattdawkins avatar Mar 02 '21 15:03 mattdawkins

@mattdawkins could we solve this with a new pipeline that simply concatenates the two pipelines in question? That would be a very general solution to this and other problems like it.

waxlamp avatar Jan 12 '22 16:01 waxlamp

Are you using "pipeline" in the generic sense or the "VIAME pipeline" sense? The latter is not possible, because transcoding is not a viame pipeline process, it's handled by dive.

From the user's perspective, no matter how we implement it, there needs to be a new option during ingest to invoke this operation.

subdavis avatar Jan 12 '22 16:01 subdavis

I meant "VIAME pipeline", with the hope that we could solve this completely out of band. But since that's not possible, we can look at a way of doing it in software.

waxlamp avatar Jan 12 '22 23:01 waxlamp

We could make a pipeline that transcodes videos into whatever format you need, and concurrently runs the analytics on the data.

It's a little cumbersome from the GUI perspective though and I hit this the other night (I wanted to start a run before I went to bed on a video, but had to wait 30 minutes for it to transcode before I could set up a run on the data)

mattdawkins avatar Apr 12 '22 14:04 mattdawkins

Also that would make a lot of duplicate .pipe files for no reason

mattdawkins avatar Apr 12 '22 14:04 mattdawkins

Might be able to make a KWIVER CLI option that adds two pipes together to prevent having to duplicate pipes, then it would be a CLI option to output transcoded videos to some file.

Though I think a pipeline dropdown in the import option would be a little cumbersome as opposed to using the main data display / data list to launch pipelines on multiple sequences.

mattdawkins avatar Apr 19 '22 14:04 mattdawkins

That said, a pipeline dropdown on dataset import might be useful for image enhancement pipelines, something we'd want to add for those (I could see listing them first then processing pipelines for immediate run, longer term)

mattdawkins avatar Apr 19 '22 15:04 mattdawkins