BiLLM icon indicating copy to clipboard operation
BiLLM copied to clipboard

Published 20 hours ago • verified publisher icon WhereIsAI
→

Metadata

Tool for converting LLMs from uni-directional to bi-directional by removing causal mask for tasks like classification and sentence embeddings. Compatible with 🤗 transformers.

Results 3 BiLLM issues
Sort by recently updated
recently updated
newest added

Bugfix/bi attention mask

- fix bug: attention_mask=None - upgrade transformers

SeanLee97 avatar SeanLee97

Example for sequence classification

1 comment

Hi, Thank you for really nice piece of work. How can i do sequence classification with this? since you only provide example for NER.

Mahmedturk avatar Mahmedturk

feature

Interaction of padding and bidirectional mask

Hi, Thanks for sharing this very interesting work. I had a question about how the bidirectional attention mask is implemented [here](https://github.com/WhereIsAI/BiLLM/blob/main/src/billm/modeling_mistral.py#L141) Based on this implementation, it seems like even the...

vaibhavad avatar vaibhavad

← Metadata

18
Stars
4
Forks
Watchers

Owner

verified publisher icon WhereIsAI

Metadata

Tool for converting LLMs from uni-directional to bi-directional by removing causal mask for tasks like classification and sentence embeddings. Compatible with 🤗 transformers.

Back