sglang
sglang copied to clipboard
use `torch.clamp`
Motivation
Motivated by the note in code, we should use torch.clamp cc @merrymercy if you could please look.
https://pytorch.org/docs/stable/generated/torch.clamp.html
Modifications
use torch.clamp for val
The
torch.minimumfunction calculates the element-wise minimum between the seq_lens tensor and a scalar tensorself.sliding_window_size + 1. Now, no value inpaged_kernel_lens_tmpexceedsself.sliding_window_size + 1. Values inseq_lensthat are already less than or equal to this limit remain unchanged.
Checklist
- [x] Format your code according to the Code Formatting with Pre-Commit.
- [ ] Add unit tests as outlined in the Running Unit Tests.
- [ ] Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
- [ ] Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
- [ ] For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
- [x] Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.