Michael McCandless

OK I made this change and kicked off `infer_token_vectors_cohere.py` again and it looks to at least be running...: ``` diff --git a/src/python/infer_token_vectors_cohere.py b/src/python/infer_token_vectors_cohere.py index 5c350df..5027eb2 100644 --- a/src/python/infer_token_vectors_cohere.py +++ b/src/python/infer_token_vectors_cohere.py...

OK hmm scratch that, I see from the already loaded features that `Dataset` thinks these `emb` vectors are already `float32`: ``` features: {'id': Value(dtype='int32', id=None), 'title': Value(dtype='string', id=None), 'text': Value(dtype='string',...

OK! Now I think the issue is in `np.array` -- I think we have to give it a preferred data type, else, it seems to be casting the Dataset's float32...

OK the above change seemed to have worked (I just pushed it)! I now see these vector files: ``` -rw-r--r-- 1 mike mike 80G Mar 28 12:57 cohere-wikipedia-768.vec -rw-r--r-- 1...

OK I think the next wrinkle here is ... to fix `SearchPerfTest` to use the pre-computed Cohere query vectors from `cohere-wikipedia-queries-768.vec`, instead of attempting to do inference based on the...

I was finally able to index/search using these Cohere vectors, and the profiler output is sort of strange: This is CPU: ``` PROFILE SUMMARY from 44698 events (total: 44698) tests.profile.mode=cpu...

Here's the `perf.py` I ran (just A/A): ``` import sys sys.path.insert(0, '/l/util/src/python') import competition if __name__ == '__main__': sourceData = competition.sourceData('wikimediumall') sourceData.tasksFile = '/l/util/just-vector-search.tasks' comp = competition.Competition(taskRepeatCount=200) #comp.addTaskPattern('HighTerm$') checkout =...

More thread context for the CPU profiling: ``` PROFILE SUMMARY from 10264 events (total: 10264) tests.profile.mode=cpu tests.profile.count=50 tests.profile.stacksize=8 tests.profile.linenumbers=false PERCENT CPU SAMPLES STACK 12.59% 1292 jdk.incubator.vector.FloatVector#reduceLanesTemplate() at jdk.incubator.vector.Float256Vector#reduceLanes() at org.apache.lucene.internal.vectorization.PanamaVectorUtilSupport#dotProductBody()...

I have not looked closely but this sounds very cool!!

Phew, this is a spooky exception! I think it means that the same term was fed to the FST Builder twice in row. FST Builder in general can support this...