dive
dive copied to clipboard
Kitware
Run KPF processing in worker task
Should run KPF generation in a worker task cause it's hefty.
This actually gets into a "race" condition if a video is uploaded with KPF: the video and the KPF processing must complete before the dataset can be marked for annotation. How do we handle this?
CC @AlmightyYakob and @mattrix27
We could have an endpoint for a folder that returns if it's currently being processed or not (which would just run a check on jobs), since the post-process job could conceivably take long on other tasks too, like image/video conversion, if the files are large and/or there's a lot of them.
Then on the front-end we could just have a spinning icon or something next to a folder (instead of annotate), that indicates that it's still being processed.
Just for clarification, do we still want the kpf generation to be a worker task?
IDK, maybe it's fine like it is.
Should we wait until we find a case where it takes more than a couple seconds?
When I run it on my test data, it takes roughly ~20 seconds, not sure if that's long enough. It definitely takes longer for the video to convert, but wasn't sure if there was a ever a case where the conversion would be the bottleneck
In that case, it definitely needs to be a worker job. I haven't seen anything longer than 3-4 seconds, but I haven't tried to find anything bigger.
When I run it on my test data, it takes roughly ~20 seconds, not sure if that's long enough. It definitely takes longer for the video to convert, but wasn't sure if there was a ever a case where the conversion would be the bottleneck
I'm using the same data as @mattrix27, so this is to be expected, but I also have similar load times of ~20 seconds.
