ai-shell icon indicating copy to clipboard operation
ai-shell copied to clipboard

Worked a few times, then it stopped

Open jkolyer opened this issue 11 months ago • 8 comments

I installed it and ran it for a few queries which worked. Then suddenly it stopped working, and now interprets all input as script. It gives an explanation about creating a script, which is not what was asked.

$ ai 'list all log files'


┌  AI Shell
│
◇  Your script:

script

jkolyer avatar Mar 13 '24 06:03 jkolyer

Same here

molove avatar Apr 08 '24 10:04 molove

ah, i think this might be issues with GPT 3. could you try using GPT 4 (e.g. gpt 4 turbo) via ai config command and lmk if this improves?

steve8708 avatar Apr 10 '24 16:04 steve8708

Same here

It seems to have started working again for me now. Unfortunately I can't remember the exact working that I had previously used that just produced "script"

molove avatar Apr 10 '24 20:04 molove

i only have gpt3 access. please update the documentation accordingly.

On Wed, Apr 10, 2024 at 1:22 PM molove @.***> wrote:

Same here

It seems to have started working again for me now. Unfortunately I can't remember the exact working that I had previously used that just produced "script"

— Reply to this email directly, view it on GitHub https://github.com/BuilderIO/ai-shell/issues/105#issuecomment-2048372880, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADWXLERXSYQLFE3TUIEKCLY4WNQZAVCNFSM6AAAAABETQ32S6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBYGM3TEOBYGA . You are receiving this because you authored the thread.Message ID: @.***>

--

@.*** (c) 415 699 2175

jkolyer avatar Apr 11 '24 08:04 jkolyer

I only have GPT 3.5 too. It's sometimes working and sometimes not now for me

molove avatar Apr 11 '24 16:04 molove

oh wow I thought GPT 4 was GA. you mean in openai dashboard/API you don't have access to any GPT 4 models?

steve8708 avatar Apr 11 '24 16:04 steve8708

I just have a pay as you go account credited with a fixed amount rather than a subscription which limits me to GPT 3.5. I'm just a user who uses Linux as my main OS who occasionally need to tinker under the hood. I can't justify $20/month for something I rarely use

molove avatar Apr 11 '24 16:04 molove

darn, I see. the problem we're finding is that GPT 3.5 has seemingly gotten worse with time at following (sometimes basic) instructions, which throws off our parsing logic.

I tried to address it previously but the lack of determinism around GPT 3.5 has been a challenge. GPT 4 handles this much better and more consistently

I would suggest in the short term, a PR to make modifications/improvements to the logic about parsing the command out of the response from GPT would be the best solution

steve8708 avatar Apr 15 '24 15:04 steve8708