mycli icon indicating copy to clipboard operation
mycli copied to clipboard

Reduce size of LLM prompts + cache per-schema context

Open rolandwalker opened this issue 2 months ago • 0 comments

Description

  • truncate text/binary sample data fields to 1024 characters (or smaller if judged to be needed)
  • truncate entire tables from schema representation if the representation is very large
  • for latency improvement, cache sample data and schema representation, passing the dbname in both cases to invalidate the cache if changing the db
  • add separate progress message when generating sample data

The target_size values are chosen somewhat arbitrarily, and it could be nice for the user to be able to control these.

We could also apply final size limits to the prompt string, though meaning-preserving truncation at that point is harder.

Addresses #1348.

Untested at the time of writing.

Checklist

  • [x] I've added this contribution to the changelog.md.
  • [x] I've added my name to the AUTHORS file (or it's already there).
  • [x] I ran uv run ruff check && uv run ruff format && uv run mypy --install-types . to lint and format the code.

rolandwalker avatar Oct 25 '25 22:10 rolandwalker