obsidian-api icon indicating copy to clipboard operation
obsidian-api copied to clipboard

Bug: `requestUrl()` and server-sent events

Open ofalvai opened this issue 1 year ago • 4 comments

I'm using Obsidian's requestUrl() helper to bypass CORS restrictions. The API I'm calling is Anthropic's standard chat API that streams AI response chunks using server-sent events. I'm only focusing on desktop only for now, so I assume requestUrl() calls a Node.js or Electron networking API.

I noticed that even though I have a working API request, the timing of events doesn't feel right.

Given this code (which I tried to make as short as possible, sorry!):

console.log("Before requestUrl")
const response = await requestUrl({
	url: "https://api.anthropic.com/v1/messages",
	method: "POST",
	contentType: "application/json",
	headers: {
		"anthropic-version": "2023-06-01",
		"x-api-key": this.apiKey,
		"accept": "text/event-stream"
	},
	body: JSON.stringify({
		stream: true,
		model: this.model,
		system: "Dummy system message",
		messages: [{
			role: "user",
			content: "Dummy user message",
		}],
	}),
})
console.log("After requestUrl")

// Creating a Web API Response because `events()` expects a Response object
const nativeResponse = new Response(response.arrayBuffer, {
	headers: response.headers,
	status: response.status,
})
console.log("After nativeResponse")

// `events()` is a thin wrapper around SSEs: https://github.com/lukeed/fetch-event-stream
const stream = events(nativeResponse)
console.log("After events")

for await (const event of stream) {
	console.log("Stream event", event)
}

...what I see is that await requestUrl() blocks until the last server-sent event is received. In practice, streaming the response chunks doesn't feel any different than using the non-streaming API because the entire response appears at once (after a long delay).

I don't think this is a problem with the fetch-event-stream wrapper because the blocking happens earlier, between the log lines Before requestUrl and After requestUrl. The remaining log lines appear instantly and at the same time.

I'm not sure this is a bug in itself, maybe I'm doing something wrong here? I would appreciate any help or guidance about how to use requestUrl() or what it's doing behind the scenes. Thank you!

ofalvai avatar May 15 '24 15:05 ofalvai