Add retry mechanism for transient API errors (503, 429, timeouts)

[bharath03-a]

Description

Implements a retry mechanism to handle transient API errors like 503 "The model is overloaded", 429 rate limits, and timeouts. This prevents the loss of all processing work when encountering temporary service issues.

Changes Involved

• Multi-level retry strategies
• Exponential backoff with jitter to prevent thundering herd problems
• Configurable retry parameters for user customization
• Fallback mechanism: batch failures fall back to individual chunk processing
• Comprehensive error detection for transient vs permanent errors

API Changes

New optional parameters in extract() function:

• retry_transient_errors: bool = True
• max_retries: int = 3
• retry_initial_delay: float = 1.0
• retry_backoff_factor: float = 2.0
• retry_max_delay: float = 60.0

Files Changed

• langextract/retry_utils.py - New retry utility module
• langextract/extraction.py - Added retry parameters to extract()
• langextract/annotation.py - Integrated retry logic in processing
• langextract/providers/gemini.py - Applied retry decorator
• tests/retry_utils_test.py - New test suite
• tests/annotation_test.py - Added retry integration tests

How Has This Been Tested?

• Unit tests for retry utility functions
• Integration tests for retry parameter passing
• Error detection and retry behavior validation
• Manual testing with 503 error scenarios

Checklist

• [x] Code follows the project's style guidelines
• [x] Self-review of the code has been performed
• [x] Tests have been added that prove the fix is effective
• [x] New and existing unit tests pass locally
• [x] Changes are backward compatible

Fixes #240

[github-actions[bot]]

❌ Infrastructure File Protection

This PR modifies protected infrastructure files:

• .github/workflows/ci.yaml (12 changes)
• .github/workflows/validate_pr_template.yaml (6 changes)

Only repository maintainers are allowed to modify infrastructure files (including .github/, build configuration, and repository documentation).

Note: If these are only formatting changes, please:

1. Revert changes to .github/ files
2. Use ./autoformat.sh to format only source code directories
3. Avoid running formatters on infrastructure files

If structural changes are necessary:

1. Open an issue describing the needed infrastructure changes
2. A maintainer will review and implement the changes if approved

For more information, see our Contributing Guidelines.

[bharath03-a]

❌ Infrastructure File Protection

This PR modifies protected infrastructure files:

• .github/workflows/ci.yaml (12 changes)
• .github/workflows/validate_pr_template.yaml (6 changes)

Only repository maintainers are allowed to modify infrastructure files (including .github/, build configuration, and repository documentation).

Note: If these are only formatting changes, please:

1. Revert changes to .github/ files
2. Use ./autoformat.sh to format only source code directories
3. Avoid running formatters on infrastructure files

If structural changes are necessary:

1. Open an issue describing the needed infrastructure changes
2. A maintainer will review and implement the changes if approved

For more information, see our Contributing Guidelines.

Fixed the issues!

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 1 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 1 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 1 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 1 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 1 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 4 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 4 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 7 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 9 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 11 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 11 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

[github-actions[bot]]

⚠️ Branch Update Required

Your branch is 11 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.