FastDeploy
FastDeploy copied to clipboard
[Bug fix] Sync status for caching output cache
Motivation
缓存输出 token 时,确认同步 token processor 和 resource manager 中关于请求的状态,当前请求尚未被抢占。
Modifications
Usage or Command
Accuracy Tests
Checklist
- [ ] Add at least a tag in the PR title.
- Tag list: [
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]] - You can add new tags based on the PR content, but the semantics must be clear.
- Tag list: [
- [ ] Format your code, run
pre-commitbefore commit. - [ ] Add unit tests. Please write the reason in this PR if no unit tests.
- [ ] Provide accuracy results.
- [ ] If the current PR is submitting to the
releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.
Thanks for your contribution!
Codecov Report
:x: Patch coverage is 28.57143% with 10 lines in your changes missing coverage. Please review.
:warning: Please upload report for BASE (develop@21fa2ba). Learn more about missing BASE report.
Additional details and impacted files
@@ Coverage Diff @@
## develop #5584 +/- ##
==========================================
Coverage ? 62.60%
==========================================
Files ? 329
Lines ? 41710
Branches ? 6371
==========================================
Hits ? 26114
Misses ? 13621
Partials ? 1975
| Flag | Coverage Δ | |
|---|---|---|
| GPU | 62.60% <28.57%> (?) |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.