disco icon indicating copy to clipboard operation
disco copied to clipboard

Add an LLM generation UI

Open JulienVig opened this issue 1 year ago • 3 comments

After training an LLM such as for the wikitext task, it would be expected to be able to prompt the model and generate some text. Right now, no such UI is available, we can only test and predict on connected text files.

We can implement a simple text box where users can input free text, tweak the generation parameters and see the model's completion in real time in another text field below.

JulienVig avatar Jun 13 '24 10:06 JulienVig

I'm having an issue with generating with the gpt-tfjs model. somewhere inside the implementation, it is failling in deep tensorflow code.

TypeError: info is undefined
    moveData engine.ts:426
    get backend.ts:55
    reshape4 Reshape.ts:44
    gatherV22 GatherV2.ts:57
    kernelFunc engine.ts:647
    runKernelFunc engine.ts:713
    [...]
    predictLoop training.ts:1035
    predict training.ts:1116
    idxNext model.js:160
    [...]
    generateOnce model.js:152
    generate model.js:140
    generate index.js:71
    generate TextInput.vue:70

because of the weird way that gpt-tfjs is coded (hijacking tfjs internals, redoing a bunch of tfjs), I'm not really able to find where and how it is triggered. maybe @JulienVig you knows a bit more on that as you are the one who lastly tried to fix some of its issues. as previously stated, this gpt-tfjs code is bad. it will need to be rewritten or changed with OONX once we have that. putting this feature on hold in the meantime (code on 684-add-chat-with-llm-tharvik branch)

tharvik avatar Jun 19 '24 09:06 tharvik

Just looking at the stack trace I don't know what's the error, will look into it when it becomes a priority.

JulienVig avatar Jun 24 '24 12:06 JulienVig

after hitting TypeError: info is undefined elsewhere, it seems that it was due to vue's proxy wrapping of model which do not behave nicely with it. toRaw before training with it should work around it

tharvik avatar Aug 27 '24 20:08 tharvik