qihqi

## Fix the Op info test for `new_empty .. nn.functional.adaptive_avg_pool2d` 1. Find the lines 193 to 197 of [test_ops.py](test/test_ops.py) and remove `new_empty` .. `nn.functional.adaptive_avg_pool2d` from `skip_list` 2. Run op_info test...

## Fix the Op info test for `nanquantile .. native_layer_norm` 1. Find the lines 188 to 192 of [test_ops.py](test/test_ops.py) and remove `nanquantile` .. `native_layer_norm` from `skip_list` 2. Run op_info test...

## Fix the Op info test for `min .. nanmedian` 1. Find the lines 183 to 187 of [test_ops.py](test/test_ops.py) and remove `min` .. `nanmedian` from `skip_list` 2. Run op_info test...

## Fix the Op info test for `matrix_exp .. median` 1. Find the lines 178 to 182 of [test_ops.py](test/test_ops.py) and remove `matrix_exp` .. `median` from `skip_list` 2. Run op_info test...

## Fix the Op info test for `masked.softmax .. masked.var` 1. Find the lines 173 to 177 of [test_ops.py](test/test_ops.py) and remove `masked.softmax` .. `masked.var` from `skip_list` 2. Run op_info test...

As reported by @tengomucho Currently there are a few issues with prefill / generate implemention: 1. Prefill does not use `self._sample` to do sampling. 2. Prefill returns a token, so...

[Feature Request] Per request sampling params

3

Currently sampling params such as temperature are set as commandline flags in when the server starts. It would be nice for each request to pass in the sampling params instead.

Recently we added a new cli `jpt` (https://github.com/google/jetstream-pytorch/pull/178) that massively simplified the command line args the user need to specify. However, there are other commandline args that are optional but...