Mika Senghaas comments

Results 6 comments of


                                            Mika Senghaas

Proposal for default fallback for invalid tool call args in `env_response`

+1 on this -- making this high prio internally to have more graceful handling of invalid tool call formats

Display running average of metrics during rollout generation + scoring

probably completion len yea

Unexpected argument 'user_agent'

yea, this was a breaking change in our core `prime` package. upgrading the package should resolve the issue!

Support env mixing with interleave_datasets

> @mikasenghaas can we remove the `probabilities` field? then should be good to merge. done!

Pydantic doesn't play nicely with OAI multimodal content + tool calls because of`typing.Iterable` lazy validation

okay, so looked a bit further and looks like this is how Pydantic handles fields that are typed as `typing.Iterable`. It [seems to be used for infinite generators](https://docs.pydantic.dev/1.10/usage/types/#infinite-generators) (and as...

Pydantic doesn't play nicely with OAI multimodal content + tool calls because of`typing.Iterable` lazy validation

Update: This also causes issues with tool calls as they are also typed with `typing.Iterable`