perf: migrate to async architecture with Quart and optimize resource …
When submitting this pull request, I confirm the following (please check the boxes):
-
[x] I have read and understood the Contributor Guidelines.
-
[x] I have checked that there are no duplicate pull requests related to this request.
-
[x] I have considered, and confirm that this submission is valuable to others.
-
[x] I accept that this submission may not be used and the pull request may be closed at the discretion of the maintainers.
Fill in the PR content:
Summary
This PR refactors the Python RPC server to use async/await architecture, significantly improving performance, reducing memory and CPU usage, and enabling better concurrent request handling.
Changes
Framework Migration
- Replaced Flask with Quart: Migrated from synchronous Flask to async-compatible Quart framework, maintaining API compatibility while enabling async request handling
- Added uvloop support: Integrated uvloop event loop policy for Linux/Mac systems, providing up to 2-4x performance improvement over the default asyncio event loop
Async Conversion
- HTTP Downloaders: Converted
HttpDownloaderandHttpMultiLinkDownloaderto fully async operations usingasyncio.to_thread()for blocking I/O - Torrent Downloader: Refactored
TorrentDownloaderto use async methods with proper thread pool execution - Image Processor: Made
ProfileImageProcessorasync with proper context managers for memory-efficient image handling
Performance Optimizations
- Concurrent Operations: Implemented
asyncio.gather()for parallel status checks in multi-link downloads - Optimized Locking: Improved lock usage to prevent serialization of concurrent operations
- Lazy Initialization: Added lazy initialization for aria2p API clients to reduce startup overhead
- Memory Optimization:
- Converted trackers list to immutable tuple
- Proper cleanup of completed downloads
- Context managers for image processing
Code Quality Improvements
- Type Hints: Added comprehensive type hints throughout all modules
- Error Handling: Improved exception handling with silent failures where appropriate
- Resource Management: Better cleanup and resource management patterns
Benefits
- Performance: Significantly improved request handling throughput with async architecture
- Resource Usage: Reduced memory footprint and CPU usage through optimizations
- Scalability: Better handling of concurrent requests and downloads
- Maintainability: Improved code quality with type hints and better structure
Testing
- All existing endpoints maintain API compatibility
- Async operations properly handle concurrent requests
- Memory usage optimized for long-running operations
Dependencies
- Updated
requirements.txt:- Replaced
flaskwithquart - Added
uvloop(Linux/Mac only, conditional import)
- Replaced
Files Changed
python_rpc/http_downloader.py- Full async conversionpython_rpc/http_multi_link_downloader.py- Async with concurrent status checkspython_rpc/torrent_downloader.py- Async operations with optimized trackerspython_rpc/profile_image_processor.py- Async image processingpython_rpc/main.py- Complete rewrite using Quart with uvlooprequirements.txt- Updated dependencies
Quality Gate passed
Issues
4 New issues
0 Accepted issues
Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code
@kleeedolinux apologies for the huge delay on this, but could you please fix the conflicts? I'll be testing it ASAP, this looks great. Thank you.