🐛 BUG: duplex option is not set in `Proxy.fetch`

[yusukebe]

Which Cloudflare product(s) does this pertain to?

Wrangler core, Miniflare

What version(s) of the tool(s) are you using?

Wrangler 3.27.0

What version of Node are you using?

20.10.0

What operating system and version are you using?

Mac Sonoma 14.3.1

Describe the Bug

Observed behavior

This bug, or something like a bug, occurred when using getPlatformProxy() and @cloudflare/ai together.

Script:

import { getPlatformProxy } from 'wrangler';
import { Ai } from '@cloudflare/ai';

const proxy = await getPlatformProxy();
const ai = new Ai(proxy.env.AI);
//...
proxy.dispose();

Error message:

$ tsx src/index.mts
/Users/yusuke/work/c/fetch-duplex/node_modules/wrangler/wrangler-dist/cli.js:29572
            throw a;
            ^

TypeError: RequestInit: duplex option is required when sending a body.
    at new Request (/Users/yusuke/work/c/fetch-duplex/node_modules/undici/lib/fetch/request.js:491:15)
    at Request (/Users/yusuke/work/c/fetch-duplex/node_modules/miniflare/src/http/request.ts:34:3)
    at ProxyStubHandler.#fetcherFetchCall (/Users/yusuke/work/c/fetch-duplex/node_modules/miniflare/src/plugins/core/proxy/client.ts:594:19)
    at ProxyStubHandler.#call (/Users/yusuke/work/c/fetch-duplex/node_modules/miniflare/src/plugins/core/proxy/client.ts:501:52)
    at Proxy.fetch (/Users/yusuke/work/c/fetch-duplex/node_modules/miniflare/src/plugins/core/proxy/client.ts:484:25)
    at InferenceSession.run (/Users/yusuke/work/c/fetch-duplex/node_modules/@cloudflare/ai/src/session.ts:72:28)
    at Ai.run (/Users/yusuke/work/c/fetch-duplex/node_modules/@cloudflare/ai/src/ai.ts:113:44)
    at <anonymous> (/Users/yusuke/work/c/fetch-duplex/src/index.mts:7:25)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v20.10.0
error Command failed with exit code 7.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Adding duplex:'half' to fetch option in the @cloudflare/ai solves the problem. So, the option duplex:'half' is not set in the Proxy.fetch (I don't know if it is correct to say that) fetch option.

Expected behavior

Does not throw the error.

Please provide a link to a minimal reproduction

https://github.com/yusukebe/get-platform-proxy-cloudflare-ai

Please provide any relevant error logs

No response

[yusukebe]

I've talked with @mrbbot , he pointed out that we can fix this here:

https://github.com/cloudflare/workers-sdk/blob/a751489f2530bc7a7adec1167319cd145f080812/packages/miniflare/src/plugins/core/proxy/client.ts#L601

[penalosa]

@yusukebe This is fairly tricky, and I'm not sure that's necessarily the right place to solve this. The issue here is that the code for @cloudflare/ai is running in Node.js, when it's designed to be run in Workers. Adding duplex: "half" to the @cloudflare/ai SDK fixes the immediate problem, but potentially breaks that library in other ways (I'm not even sure if that field is supported in Workers)—and given the target for that library is Workers, it's not guaranteed to work properly in Node.js at all.

@G4brym do you have any thoughts here?

cc @dario-piotrowicz for visibility

[mrbbot]

I could be wrong, but I think adding { duplex: "half" } if request has a non-null body here will do the trick: https://github.com/cloudflare/workers-sdk/blob/a751489f2530bc7a7adec1167319cd145f080812/packages/miniflare/src/plugins/core/proxy/client.ts#L601 Given Workers doesn't have a duplex option AFAIK, we'd expect any .fetch() call to behave as though it were set. So adding it to that line would change the default behaviour to be in line with Workers.

[G4brym]

If adding { duplex: "half" } to @cloudflare/ai solves the issue, we can try to add it in a pre-release to see if any other issues show up We are doing a major revamp in @cloudflare/ai, so we are also going to be doing a lot of tests, maybe now is the best opportunity to do that

[yusukebe]

Hi. Thanks for the comments and the PR!

I don't know if the test is necessary or not, but I would like the fix for #5114.

[penalosa]

@yusukebe Does the linked PR fix the issue for you?

[yusukebe]

@penalosa

Yes!

[yusukebe]

Hi @mrbbot @penalosa @G4brym !

I've tried it with the new version 1.1.0 of @cloudflare/ai with getPlatformProxy(), it works well!

I'm not sure which commit fixed this issue (maybe @G4brym added?) , duplex: 'half' option is passing to fetch in the compiled JavaScript source of @cloudflare/ai.

So, my problem - I could not use @cloudflare/ai and getPlatformProxy() together - is fixed now. I'll leave it to you to decide what to do with PR #5114. But if it is not needed, I might be able to close this issue.

[penalosa]

Closing this for now since it seems @cloudflare/ai is fixed, and in any case the AI binding is now implemented in the runtime.