weave icon indicating copy to clipboard operation
weave copied to clipboard

Prototype: Nicer Python calls fetch API

Open shawnlewis opened this issue 8 months ago • 1 comments

This is a prototype that makes it so you can do this:

image

Or more complex stuff like

client.calls('Evaluation.evaluate', filter={'output.llm_judge_accuracy.avg_support.mean': {'$gt': 2.3}})

using Tim's new mongo calls filter capability, which has a lot of power.

The server's mongo query support is currently low-level, so I implemented a handful of query transforms to make these queries nice to write (that should be equivalent to what Mongo does)

  • {'<field>', ...} style queries
  • convert python literal to {$literal: ...}
  • add {$convert, ...} calls to convert a field on one side to literal type on the other
  • short-hand $eq syntax like {'<field>': <val>}

These transforms were implemented by ChatGPT so I'm not confident in the generality of that code yet.

shawnlewis avatar May 30 '24 23:05 shawnlewis