Sarthak Langde

icon indicating a website link https://sarthaklangde.com icon indicating an email [email protected]

icon company @tenxstudio icon location Lisbon icon location Generalist navigating the deep learning landscape. Leading AI @tenxstudio, previously @stochasticai MSc. in AI from KTH & TUB.

Results 4 issues of Sarthak Langde

question: [Quantization] Which files to change to make inference faster for Q8BERT?

1 comment

I know from previous issues it is mentioned that that Q8BERT was just an experiment to measure the accuracy of quantized BERT model. But, given that the accuracy is good,...

question

Slower inference results for BLOOM fp16 on identical hardware

5 comment

Hey, Thank you for the scripts for loading checkpoints and running benchmarks. I have a strange issue that ds_inference fp16 throughput is quite slower than the results mentioned. But, the...

Simplify notebook examples

### Description Currently, the examples directory has one directory per model and then we have many notebooks with only one line changing at max. It is quite hard to keep...

documentation
good first issue

Make xTuring config file optional or let users pass model config to BaseModel loading

### Description BaseModel load does not allow for loading of weights from non-xTuring hub based models. We should either make it optional or let users provide more details of these...

enhancement