instructor-embedding icon indicating copy to clipboard operation
instructor-embedding copied to clipboard

Published 20 hours ago • verified publisher icon xlang-ai

Redundant code & Tokenize issue

Open JonathanZha47 opened this issue 1 year ago • 3 comments

  1. Seems like there's a redundant code here. image
  2. Tokenize method not working well for creating correct instruction_mask column image

JonathanZha47 avatar Jun 03 '24 10:06 JonathanZha47

For the second problem, it appeared when I was using the 2.7.0 version of the sentence_transformers.

JonathanZha47 avatar Jun 03 '24 11:06 JonathanZha47

If I'm using the 3.0.0 version of the sentence_transformers, then the local file issue still exists even if I change my code according to the #115 or #113

JonathanZha47 avatar Jun 03 '24 11:06 JonathanZha47

After you pip install the normal instructor package, try replacing the instructor.py file in the site-packages folder with the one in my fork here...let me know if this fixes the issue:

https://github.com/BBC-Esq/instructor-embedding

BBC-Esq avatar Aug 25 '24 17:08 BBC-Esq