sglang icon indicating copy to clipboard operation
sglang copied to clipboard

use `torch.clamp`

Open vincentzed opened this issue 8 months ago • 0 comments

Motivation

Motivated by the note in code, we should use torch.clamp cc @merrymercy if you could please look.

https://pytorch.org/docs/stable/generated/torch.clamp.html

Modifications

use torch.clamp for val

The torch.minimum function calculates the element-wise minimum between the seq_lens tensor and a scalar tensor self.sliding_window_size + 1. Now, no value in paged_kernel_lens_tmp exceeds self.sliding_window_size + 1. Values in seq_lens that are already less than or equal to this limit remain unchanged.

Checklist

  • [x] Format your code according to the Code Formatting with Pre-Commit.
  • [ ] Add unit tests as outlined in the Running Unit Tests.
  • [ ] Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
  • [ ] Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
  • [ ] For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
  • [x] Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

vincentzed avatar Apr 24 '25 03:04 vincentzed