pearai-app icon indicating copy to clipboard operation
pearai-app copied to clipboard

Chunk a file if the file is too big

Open nang-dev opened this issue 9 months ago • 0 comments

Context

We want to be able to tag files that are big (within reason) We should chunk the file to send to the LLM as context

Chunking: Split the code into smaller chunks that are individually within the token limit. Each chunk can be processed separately. This is useful if you need to get insights or analysis on different parts of the code.

nang-dev avatar May 23 '24 01:05 nang-dev