Enrico Shippole

I am aware of all these alternatives. We are waiting to hear back from Huggingface before the decision is made. Once we have a concrete answer from them we will...

Llama has been added to Huggingface: https://github.com/huggingface/transformers/pull/21955 The only reason to add a specific wrapper would be to include the perf improvements from cpp or gptq

> > FYI: I just submitted this pull request to integrate llama.cpp into langchain: #2242 > > Thank you very much!! > > Do you think it would be possible...

> Also, you should probably request a review from @hwchase17 as he has write access Thank you for the additional input. I think once the basic functionality is merged we...

Hi @htthYjh , The repository initially consisted of just the pre-training architecture but I am actively updating it on a daily basis. The full repository when completed will allow for...

Hi @htthYjh , I rebuilt the data loader to work locally: https://github.com/conceptofmind/LaMDA-pytorch/blob/main/lamda_pytorch/build_dataloader.py A few things you are going to have to take into consideration if you are going to use...

@hwchase17 From my initial testing setting up a custom google search API through GCP produces nearly identical scraped results when compared with serpapi. For example, here is a comparison of...

PyTorch 2.0 will automatically set the most appropriate version of attention based on your system specs. ``` All implementations are enabled by default. Scaled dot product attention attempts to automatically...

@lucidrains I forgot to mention that I used fp16 in the training above. This is likely one of the causes of numerical instability and NaN for this experiment. Since I...

@lucidrains I will update the repository and rerun the test again with fp16 enabled. I will post the new fp16 training results. I am using the character level enwiki8 for...