SDK can't download frames from a task for video uploaded with a frame step

[radybay]

Actions before raising this issue

• [X] I searched the existing issues and did not find anything similar.
• [X] I read/searched the docs

Steps to Reproduce

Create a new task, lets say the video has 452 frames. When uploading set frame step = 4. In python do: task.download_frames(frame sequence) numbers in sequence: 0, 4, ..., 112 - success numbers in sequence not divisible by 4, eg 1,2,3,5 - HTTP response body: "The frame number should be in the [0, 448] range" numbers in sequence: 116, 120, ... HTTP response body: "Incorrect requested frame number: 116"

Expected Behavior

There's two conceivable ways it should work: frame numbers specify after the step has been applied: 0,1,...,112 frame numbers specify before the step has been applied: 0,4,..., 448

Possible Solution

No response

Context

I'm trying to download the frames & annotations of all completed jobs

Environment

CVAT 2.7.0
CVAT-SDK 2.7.0

[Jamil]

Also getting this issue as well.

[Jamil]

I think the issue is here: https://github.com/opencv/cvat/blob/f52f0f3e0ea09b72bc3efdead9780ca2fb2a858c/cvat/apps/engine/views.py#L630

frame_range = range(self._start, self._stop + 1, self._db_data.get_frame_step())
        if frame not in frame_range:
            raise ValidationError(
                f'The frame number should be in the [{self._start}, {self._stop}] range'
            )

[umangapatel123]

@nmanovic, can you please assign this issue to me? I was able to reproduce the problem. The issue is due to the following code:

            elif self.type == 'frame' or self.type == 'preview':
                self._check_frame_range(self.number)

                if self.type == 'preview':
                    cache = MediaCache(self.dimension)
                    buf, mime = cache.get_local_preview_with_mime(self.number, db_data)
                else:
                    buf, mime = frame_provider.get_frame(self.number, self.quality)

                return HttpResponse(buf.getvalue(), content_type=mime)

In this code, the get_frame function retrieves the frame using the new frame number which is after applying step size (0,1,2,3 .. 112 are valid). However, in the _check_frame_range function, we are checking the frame range before applying the step size (0,4,8,12 ...448 are valid):

    def _check_frame_range(self, frame: int):
        frame_range = range(self._start, self._stop + 1, self._db_data.get_frame_step())
        if frame not in frame_range:
            raise ValidationError(
                f'The frame number should be in the [{self._start}, {self._stop}] range'
            )

To resolve this issue, we can either Perform the frame range check after applying the step size, similar to how the frame is retrieved or divide the self.number with self._db_data.get_frame_step() while retrieving the frame to get the correct frame.

[zhiltsov-max]

@umangapatel123, assigned. Please send us a PR with the fixes and tests.

[umangapatel123]

@zhiltsov-max can you please guide me on how can I generate a video for testing? is there any existing method for it?

[zhiltsov-max]

@umangapatel123, please check if any of the existing test tasks fit your needs - just run the test server with pytest --rebuild --start-services tests/python, then open the localhost:8080 in the browser, login with the credentials admin1 and this password, and investigate the available tasks. There should be several video/image tasks with frame step. If there is no such, please check if you can use a task with a set of images with frame step (probably, it also has the same problem). If no, I can suggest to use ffmpeg of opencv to generate a video from a set of frames.