askbend icon indicating copy to clipboard operation
askbend copied to clipboard

How to avoid insert same doc while rebuild?

Open Xuanwo opened this issue 2 years ago • 1 comments

I found we will insert all content during rebuild while may double the OpenAI token cost. Is it possible to avoid it?

Xuanwo avatar Apr 10 '23 02:04 Xuanwo

Now, we use $ to reduce complexity:) but it's already quite cheap. We will plan how to make incremental embedding later on.

bohutang avatar Apr 10 '23 03:04 bohutang