cvat Validation API core

trafficstars

Motivation and context

Depends on #8272 Depends on #8321

Added server API for creation of a GT job on task creation
Added server support for task creation with Honeypot (aka GT pool)

How has this been tested?

Checklist

[ ] I submit my changes into the develop branch
[ ] I have created a changelog fragment
[ ] I have updated the documentation accordingly
[ ] I have added tests to cover my changes
[ ] I have linked related issues (see GitHub docs)
[ ] I have increased versions of npm packages if it is necessary (cvat-canvas, cvat-core, cvat-data and cvat-ui)

License

[ ] I submit my code changes under the same MIT License that covers the project. Feel free to contact the maintainers if that's a concern.

Summary by CodeRabbit

New Features
- Introduced a server setting to disable media chunks on the local filesystem, enhancing configurability.
- Added tracking for the last assignee update date in quality reports, improving task management.
- Enhanced job chunk identifiers for better clarity and uniqueness.
Bug Fixes
- Resolved memory management issues and refined job assignment logic in video processing.
Documentation
- Updated API schema with new enhancements related to job management and validation processes.
Chores
- Updated package dependencies and added new configuration settings for Redis in the Helm chart.

Aug 26 '24 16:08 zhiltsov-max

[!IMPORTANT]

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

This update encompasses a range of enhancements and fixes across various components of the application. Key changes include the introduction of new parameters for job and frame handling, improvements to memory management, and updates to the quality reporting system. Additionally, several files have been reorganized and refactored for clarity and maintainability, alongside updates to testing frameworks and JSON configurations.

Changes

Files	Change Summary
changelog.d/20240812_161617_mzhiltso_job_chunks.md, changelog.d/20240819_210200_mzhiltso_validation_api.md	Introduced server settings to disable media chunks and enhancements to quality reports with new tracking features.
cvat-core/src/frames.ts, cvat-data/src/ts/cvat-data.ts	Modified `PrefetchAnalyzer` and `FrameDecoder` classes to include `startFrame` for better chunk calculations.
cvat-sdk/cvat_sdk/core/proxies/tasks.py	Added `validation_params` to the `upload_data` function for enhanced data upload capabilities.
cvat/apps/dataset_manager/bindings.py, cvat/apps/dataset_manager/formats/cvat.py	Replaced `FrameProvider` with `TaskFrameProvider` and updated frame handling logic in `dump_media_files`.
cvat/apps/engine/cache.py, cvat/apps/engine/media_extractors.py	Significant modifications to `MediaCache` and `IMediaReader` classes for improved media handling.
cvat/apps/engine/models.py, cvat/apps/quality_control/models.py	Introduced new models (`ValidationLayout`, `ValidationFrame`) and new enumeration (`ValidationMode`) for quality control.
tests/python/sdk/test_*.py	Added `restore_redis_ondisk_per_function` parameter to various setup functions for improved test management.
tests/python/shared/assets/*.json	Added new fields for quality reports and job configurations in JSON files.

🐇 "In the fields where bunnies hop,
Changes sprout and never stop.
With frames and jobs now well-defined,
A leap ahead in every kind!
Quality blooms, and settings align,
Hooray for code, so bright and fine!" 🥕

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

Aug 26 '24 16:08 coderabbitai[bot]

/check

Sep 16 '24 15:09 zhiltsov-max

:x: Some checks failed :page_facing_up: See logs here

Sep 16 '24 15:09 github-actions[bot]

Codecov Report

Attention: Patch coverage is 60.80586% with 321 lines in your changes missing coverage. Please review.

Project coverage is 74.17%. Comparing base (ff1849f) to head (45e1b4b).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8348      +/-   ##
===========================================
- Coverage    74.35%   74.17%   -0.18%     
===========================================
  Files          394      396       +2     
  Lines        42177    42768     +591     
  Branches      3896     3897       +1     
===========================================
+ Hits         31360    31724     +364     
- Misses       10817    11044     +227

Components	Coverage Δ
cvat-ui	`78.83% <66.66%> (+0.10%)`	:arrow_up:
cvat-server	`70.16% <60.76%> (-0.33%)`	:arrow_down:

Sep 19 '24 16:09 codecov-commenter

There is a corner case when Im trying to create a task with 100% honeypots. So it gives me the task with just 1 gt job and 0 regular jobs. Is there a usecase for that? maybe we shoud prohibit such case?
I dont see any changes in .rego. Will non-admin users be able to compute quality reports after this pr?
Would it be hard to improve formatting for the validation error? Like return only the stuff thats insize string=?. I suppose we have lots of cases now when we can get it. On the second thought I belive its easier to UI to parse it for better UX, we already have some handling for such cases. Maybe not really relevant for now

Overall pr seems to be working fine

Sep 27 '24 09:09 klakhov

@klakhov ,

There is a corner case when Im trying to create a task with 100% honeypots. So it gives me the task with just 1 gt job and 0 regular jobs. Is there a usecase for that? maybe we shoud prohibit such case?

Hm, I think yes, it should produce a validation error during task creation.

I don't see any changes in .rego. Will non-admin users be able to compute quality reports after this pr?

I think it will be ether in a separate PR or in the PR with allocation reports, it seems to be the most relevant place.

Sep 27 '24 10:09 zhiltsov-max

@coderabbitai review

Sep 27 '24 16:09 zhiltsov-max

:white_check_mark: Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sep 27 '24 16:09 coderabbitai[bot]

can't create a task with honeypot job and context images:

Traceback (most recent call last):
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/rq/worker.py", line 1431, in perform_job
    rv = job.perform()
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/rq/job.py", line 1280, in perform
    self._result = self._execute()
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/rq/job.py", line 1317, in _execute
    result = self.func(*self.args, **self.kwargs)
  File "/usr/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/cvat/cvat/apps/engine/task.py", line 1347, in _create_thread
    models.RelatedFile.objects.bulk_create(db_related_files)
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/django/db/models/query.py", line 803, in bulk_create
    returned_columns = self._batched_insert(
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/django/db/models/query.py", line 1831, in _batched_insert
    self._insert(
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/django/db/models/query.py", line 1805, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
    cursor.execute(sql, params)
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/bsekachev/app.cvat.ai/cvat_enterprise/.env/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "engine_relatedfile_data_id_path_a7223d1e_uniq"
DETAIL:  Key (data_id, path)=(5, /home/bsekachev/app.cvat.ai/cvat_enterprise/cvat/data/data/5/raw/context_images example/related_images/3Z2A3692_jpg/3Z2A3692.jpg) already exists.