PaLM-rlhf-pytorch

PaLM-rlhf-pytorch copied to clipboard

Encoder-Decoder

18

The follow-up research from PaLM switched in Flan-PaLM to the encoder-decoder t5 architecture. How would it be possible to also add an encoder to this implementation?

Bachstelze

Hi, first of all thanks for your work. I will definitely give it a try. I was wondering if you could share some information about the training time and which...

ejarkm

Unified reward function/model architecture for a wide range of tasks

2

I find the reward function to be the most important part of RLHF, because it is the part which mimics a human evaluator, providing instant feedback to the model. However,...

James4Ever0

Add wandb logging

2

Logs train and val loss as well as generated texts by default only when wandb available

ell-hol

huggingface -> Hugging Face

eltociear

Noob question: How can I use this model for inference?

1

PrasoonPratham

Is it possible to release a code based on jax?

2

Is it possible to release a code based on jax?

sglucas

hi, i work at a company that wants to help. We've computational power and we would like to talk more about it, is it possible?

byteunix

A few questions on training

3

Hi, I've been planning to train this model, I have a tpu pod(v3-128) through trc, which should equate to ~ 5 tb of ram and 2 tb of vram, I...

aakashrkumar

Hi, Any references to train this on my own data ?

rbhatia46