Seongmin Park

Any updates on this?

Hello. Latent **z** is passed as `past_key_values` instead of `encoder_hidden_states` of the decoder: https://github.com/seongminp/transformers-into-vaes/blob/16205c8da8731b0097d80eeca219a878e0397beb/vendor_t5.py#L134 I think `encoder_hidden_states` is a more natural choice, but using `past_key_values` required fewer modifications of `transformers`...

Just want to say thank you for this great piece of software! Makes my life much easier. I have fond memories of iTunes and PaperLib is giving me similar vibes....

FWIW I got this error when I put my model accidentally on inference mode. My peft config had `inference_mode: True`.

Temp solution: Say you have text-generation-inference running on http://1.1.1.1:8080. in src/lib/server/generateFromDefaultEndpoint.ts, change ``` { model: endpoint.url, inputs: prompt, parameters: newParameters, } ``` to ``` { model: `http://1.1.1.1:8080`, inputs: prompt, parameters:...

After mucking around a bit I found a temporary fix. The error occurs because the summary request calls the streaming endpoint instead of the async endpoint. So [this function](https://github.com/huggingface/huggingface.js/blob/208e91408e058e80bd155d2f3143addd442c51bc/packages/inference/src/tasks/nlp/textGeneration.ts#L64) throws...

My custom gpt-neox model (trained with bf16) degrades heavily in performance when loading with fp16. Would also love this option.

HI @WYejian. `T5VAE` defined in `model_t5.py` initializes an internal t5 (`T5ForConditionalGeneration`) defined in `vendor_t5.py`. I've modified `T5ForConditionalGeneration` in `vendor_t5.py` so it takes a `sampled_z` parameter: https://github.com/seongminp/transformers-into-vaes/blob/16205c8da8731b0097d80eeca219a878e0397beb/vendor_t5.py#L46 Since we don't call...

Just uploaded `generate.py`! Thanks for pointing that out. Yes. I think it'll work better if we use the same data for pretraining and finetuning, but I wanted to work with...

Hello. I followed the [calculations](https://github.com/ChunyuanLI/Optimus/blob/9871b4b824e964a5650d9f4fbddb0c139b4caf3b/code/examples/big_ae/utils.py#L1198) provided in previous research. (The link is not the original.. If you search for `calc_mi`, several VAE repos come up.)