label-studio icon indicating copy to clipboard operation
label-studio copied to clipboard

feat: Add batch training experimental feature to allow batching of multiple tasks when training with ML backend

Open shondle opened this issue 1 year ago • 10 comments

PR fulfills these requirements

  • [ ] Commit message(s) and PR title follows the format [fix|feat|ci|chore|doc]: TICKET-ID: Short description of change made ex. fix: DEV-XXXX: Removed inconsistent code usage causing intermittent errors
  • [x] Tests for the changes have been added/updated (for bug fixes/features)
  • [x] Docs have been added/updated (for bug fixes/features)
  • [x] Best efforts were made to ensure docs/code are concise and coherent (checked for spelling/grammatical errors, commented out code, debug logs etc.)
  • [x] Self-reviewed and ran all changes on a local instance (for bug fixes/features)

Change has impacts in these area(s)

(check all that apply)

  • [ ] Product design
  • [ ] Backend (Database)
  • [x] Backend (API)
  • [ ] Frontend

Describe the reason for change

(link to issue, supportive screenshots etc.) Wanted to batch multiple tasks together from database view before training model. This feature allows the user to, after turning on the experimental feature, to select multiple tasks and send it to the backend for training. Before, it was only possible to do one at a time. For most machine learning / deep learning domains, task batching is preferred to train models used for inference.

What does this fix?

(if this is a bug fix) N/A

What is the new behavior?

(if this is a breaking or feature change) After turning on experimental features, the user can select multiple tasks from the task view and send a batch of tasks to train their ML backend.

What is the current behavior?

(if this is a breaking or feature change)

What libraries were added/updated?

(list all with version changes)

Does this change affect performance?

(if so describe the impacts positive or negative) No

Does this change affect security?

(if so describe the impacts positive or negative) No

What alternative approaches were there?

(briefly list any if applicable)

This feature could be added to main software, but for testing I am currently keeping the batch training option as an experimental feature.

What feature flags were used to cover this change?

(briefly list any if applicable)

Does this PR introduce a breaking change?

(check only one)

  • [ ] Yes, and covered entirely by feature flag(s)
  • [ ] Yes, and covered partially by feature flag(s)
  • [x ] No
  • [ ] Not sure (briefly explain the situation below)

What level of testing was included in the change?

(check all that apply)

  • [ ] e2e
  • [ ] integration
  • [ ] unit

Which logical domain(s) does this change affect?

(for bug fixes/features, be as precise as possible. ex. Authentication, Annotation History, Review Stream etc.) Task Manager training

shondle avatar Nov 29 '23 14:11 shondle

Deploy Preview for label-studio-docs-new-theme ready!

Name Link
Latest commit 459c94a024e92d5d9b5654793651ea9e137ffee0
Latest deploy log https://app.netlify.com/sites/label-studio-docs-new-theme/deploys/656d16833861b30008c9a43b
Deploy Preview https://deploy-preview-5108--label-studio-docs-new-theme.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

netlify[bot] avatar Nov 29 '23 14:11 netlify[bot]

Deploy Preview for heartex-docs ready!

Name Link
Latest commit 459c94a024e92d5d9b5654793651ea9e137ffee0
Latest deploy log https://app.netlify.com/sites/heartex-docs/deploys/656d1683b8f6510008932f1b
Deploy Preview https://deploy-preview-5108--heartex-docs.netlify.app/guide/billing
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

netlify[bot] avatar Nov 29 '23 14:11 netlify[bot]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 75.96%. Comparing base (e0dd326) to head (459c94a). Report is 2095 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #5108      +/-   ##
===========================================
- Coverage    76.03%   75.96%   -0.07%     
===========================================
  Files          154      154              
  Lines        12792    12809      +17     
===========================================
+ Hits          9726     9730       +4     
- Misses        3066     3079      +13     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov[bot] avatar Nov 30 '23 01:11 codecov[bot]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

robot-ci-heartex avatar Jun 10 '24 21:06 robot-ci-heartex

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

robot-ci-heartex avatar Oct 19 '24 01:10 robot-ci-heartex

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

robot-ci-heartex avatar Dec 04 '24 02:12 robot-ci-heartex

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

robot-ci-heartex avatar Jan 20 '25 01:01 robot-ci-heartex

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

robot-ci-heartex avatar Mar 10 '25 01:03 robot-ci-heartex

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

robot-ci-heartex avatar Apr 25 '25 02:04 robot-ci-heartex

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

robot-ci-heartex avatar Jun 10 '25 02:06 robot-ci-heartex