Akash

I used the tool by a running slurm job . Whereas the actual time I observed was loading the model and timing it using time module when given a prompt...

what happened the tool inference time is not matching with the inference i am taking in real time. I have ran it on v100 gpu card of 16 gb.

yeah that worked . And also could you tell how to use this for big models with more parameters . like in my case when i try to load a...

I have made changes in the code in place of memory and have given more space. Now the model is running . But will this change alter the results from...