Billy Cao

How to reproduce T2TT result using HF?

3

I am trying the HF code and it produces poorer result than the space one (which uses this repo). How can I reproduce using HF only? HF code I am...

### Discussed in https://github.com/aliencaocao/netease_cloudmusic_discord_rpc/discussions/34 Originally posted by **chenchen7427** May 7, 2024 一个小建议 就是不知道能不能实现比如半个小时没有播放音乐 就取消discord上网易云的状态

V3.0 beta client support

1

It seems like V3.0 added address randomisation causing the pointer offsets to be random on each machine. Code for reading them already works IF the randomisation can be solved. E.g.,...

Possible to use with a VL model like LLAVA?

2

I am trying to use this project with a vision-language model like https://huggingface.co/docs/transformers/en/model_doc/llava_next but currently this repo does not support vision part of the model. I have a separate script...

RuntimeError in llava image encoding

5

When running llava 1.6 mistral 7b, i get this error: ``` RuntimeError in llava image encoding: The expanded size of the tensor (0) must match the existing size (2438) at...

RuntimeError: CUDA error: device-side assert triggered when running

7

``` [2024-03-10 10:31:21,586] [ ERROR] model_rpc.py:178 - Exception in ModelRpcClient: Traceback (most recent call last): File "/venv/lib/python3.9/site-packages/sglang/srt/managers/router/model_rpc.py", line 176, in exposed_step self.forward_step() File "/venv/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args,...

When I am generating answers on LLAVA, I am using regex to specify a template, and the last token is always either 'Yes' or 'No. One weird thing I have...

## Modified AdaFactor by ViT paper: [here!](https://arxiv.org/pdf/2106.04560) section 3.4 Actually I'm not sure if it is already implemented/possible to tweak from params with the existing Adafactor optimizer. Please let me...

Allow FP16 or other precision inference for Pipelines

19

# What does this PR do? Currently, if you pass `torch_dtype=torch.float16` or set `model=AutoModel.from_pretrained(..., torch_dtype=torch.float16)` in hope to use FP16 for inference in a `Pipeline`, it will fail because although...

Add training support for SigLIP

4

# What does this PR do? Add the sigmoid contrastive loss function of SigLIP from https://github.com/google-research/big_vision/blob/01edb81a4716f93a48be43b3a4af14e29cdb3a7f/big_vision/trainers/proj/image_text/siglip.py#L287 This will allow training/finetuning SigLIP models. Already verified to work on my own dataset....