LAVIS

LAVIS copied to clipboard

Build issues with decord for Mac with Python 3.9+

8

Decord is listed as a dependency but there is no prebuilt binaries for Mac above Python 3.8 (see issues [here](https://github.com/dmlc/decord/issues/213)). This means Mac users of LAVIS effectively have to stay...

jhwang7628

release web demo

1

when is the release?

micole66

Using albef to do image tagging

1

Hi, i want to do image tagging using albef model. I have writen the code for that but the top one tags are way off. Could you help we regarding...

cs20s030

BLIP2 VQA Questions

13

Hi, thanks for the great work on BLIP2, and also for open-sourcing the model and code! I was trying to apply 'blip_t5' with model type "pretrain_flant5xxl" to VQA settings, and...

sachit-menon

Unable to install salesforce-lavis in Python 3.9.13 Jupyter notebook

3

I am currently using Python 3.9.13 version I tried to install salesforce Lavis in Jupyter notebook using pip **pip install salesforce-lavis** But I got the below error, it says that...

innominate-dreamer

[Bug] checkpointing with momentum leads to parameters inconsistency in BLIP

1

I found that during blip coco retrieval finetuning, the `vit_grad_ckpt` is True which means checkpointing is used in VIT and `weakref` is used in replaced `forward` function https://github.com/salesforce/LAVIS/blob/main/lavis/models/vit.py#L152 However, in...

yingfhu

Possible to reduce the VRAM usage just by 10%?

1

Hello! I was a big fan already of BLIP (1) and very impressed by the improvement of BLIP 2. I have a RTX 4090 but unfortunatelly its VRAM is just...

Marcophono2

Unwanted guidance when textual content is overlaid in the image

1

Hi, was exploring with both the decoder-only and encoder-decoder models for captioning and VQA task, and I realised that the model tends to be misguided with the presence of text...

adrielkuek

Currently use default parameters of AdamW: https://github.com/salesforce/LAVIS/blob/main/lavis/runners/runner_base.py#L103. This has to be made more flexible. Also, it would be good to have a standalone optimizer class, and related registry etc.

dxli94

Currently, when using iteration-based runner, checkpointing always happens after each inner epoch, thus coupling with the number of iterations per inner epoch. Ideally, the frequency to checkpoint/logging/ lr scheduling should...

dxli94