Add "strict" interface to `replicate.run()`
This PR implements a new strict option that can be passed to run() to coerce the output into the same data types as defined by the model schema.
const stream = replicate.run("my-org/my-llm", {strict: true, input: { ... }});
for await (const chunk in stream) {
...
}
const files = replicate.run("my-org/my-gen-img", {strict: true, input: { ... }});
for await (const image in files) {
const img = Object.createURLObject(await file.blob());
}
const file = replicate.run("my-org/my-model", {strict: true, input: { ... }});
const data = await file.arrayBuffer();
The main two benefits here are that:
- Iterators get converted into actual async iterators instead of just arrays which means the
stream()method can be deprecated in favor of a single interface. This opens up opportunities for streaming other types of output too. Path&Fileoutputs are backed by aBloblike representation that can be used to download the data opening up opportunities to stream files directly in the client. Note, that we don't useBlob,FileorArrayBufferdirectly because it's likely the client will want the opportunity to decide whether to actually download the file or not. Similar to howbodyonResponseis not consumed until an async read method is called. Feedback on this appreciated.
[!NOTE] The intention here is that once this code is merged and validated we migrate the SDKs to use this as the default interface to
run()(a breaking change). Thestrictflag is a way of soft launching the feature. The namestrictwas chosen as in JavaScript land "use strict" and TypeScript--strictmode are existing functionality for tightening up interfaces.
The need to download the schema for the version is currently unfortunate, I'm also looking at extending the Prediction interface to include input_schema and output_schema to reduce this additional request.
Implementation
The bulk of the code exists in the coerceOutput() function, I think it implements the bulk of the Cog Output interface, but I may have missed some edge cases.
TODO
- [ ] Currently there is no support for
enumtypes. - [ ] Currently there is no support for date strings.