Is there a way to see response data for streaming API calls (eg. endpoints meant for receiving Server sent events)

[gamerkhang]

Currently you do not see the response data for an API call until it's completed However, there are API calls that are meant to stay active to intercept server sent events, where the response data is continuously streamed

Is there a way to see this information on Proxyman?

[NghiaTranUIT]

We implemented this feature in the past, but it didn't work so well, so we completely removed it 😿

Can you elaborate on what type of streaming API you'd like to check? (Content-Type?)

[ivanmoskalev]

I'm guessing server-sent events (text/event-stream)

[wesbos]

This would be a neat feature to have - as streaming becomes more popular in the browser, there isn't a single tool I've found that will allow you to see the streamed request as it's coming in. All wait for the request to close before showing the entire payload of data

A common use case right now is many of these GPT chat apps will stream the response in from OpenAI. Many use Web Streams (https://developer.mozilla.org/en-US/docs/Web/API/Streams_API) and others use server sent events.

[NghiaTranUIT]

I guess I can support the Streaming Body by looking at the Response Header:

• If the Content-Length is absent
• Or Transfer-Encoding: chunked

Let me play around and send you a Beta build 👍

Reference: https://gist.github.com/CMCDragonkai/6bfade6431e9ffb7fe88

[SOVRON]

Any update on this? I too am trying to intercept our ChatGPT stream api request but Proxyman Mac app shows nothing? Thanks

[farmisen]

I would also be super interested by that feature - trying to debug our in house sse streamed events and being able to see them as they come instead of at once when the last one is sent would help a ton. I'll definitely be able to help QA that feature if needed.

[reubn]

Currently facing this issue as well

[ChristianWeyer]

Oh yeah, this is a super helpful feature @NghiaTranUIT - any updates on this?

Thanks!

[NghiaTranUIT]

@ChristianWeyer not yet 😢 I tried to implement it but it breaks our current flow and doesn't meet our requirements. Thus, we postpone it until we find a better solution.

For example:

• For a single request/response, Proxyman receives a lot of chunk in a very short time (millisecond) -> causing the UI to update too many times -> Lag and unresponsive.

[ChristianWeyer]

Thanks for getting back with the details @NghiaTranUIT - do you know of any similar HTTPS debugging proxy tool running on macOS that can handle response streaming?

[NghiaTranUIT]

@ChristianWeyer you can use Charles Proxy. However, it's hard to set and you might follow some tutorials on Google 👍

[ChristianWeyer]

Charles is too slow and cumbersome... 😅

[NghiaTranUIT]

Good news everyone 🎉

• Proxyman now supports ServerSentEvent and displays the body as soon as there is new stream data
• Work if it's SSE with Content-Type: text/event-stream
• Beta build: https://download.proxyman.io/beta/Proxyman_5.0.0_Support_SSE_v2.dmg

Video

https://github.com/ProxymanApp/Proxyman/assets/5878421/fba011b2-576e-4fcd-9d34-a5e489d19400

---

@ChristianWeyer @reubn @farmisen @SOVRON please give it a try and share with me the result 👍 I appreciate it 🙇

[reekystive]

It's working! 🎉

But there's a issue if scripting is enabled. I use scripts to add custom headers and don't change HTTP body data. With scripting enabled, SSE data is showing as stream in Proxyman but Chrome isn't receiving any data from Proxyman until the request is done. Chrome received all the SSE data at once.

Can we have an option to tell Proxyman a script will not modify HTTP body, so it doesn't have to wait for the entire request to end, but instead returns the data to the client in realtime?

[NghiaTranUIT]

@reekystive I'm not sure how to implement the Scripting with SSE yet.

Currently, When a request matches with Scripting/Breakpoint, the script will be executed when the body is fully received -> So, we can modify the body (response.body) -> Then, it writes entire HTTP Response to the client.

[NghiaTranUIT]

@reekystive I'm working on this change. May I ask:

• Do you use Scripting to modify the head of SSE Request or Response part?

[reekystive]

@NghiaTranUIT I only modify the request header with scripts to test APIs in production and test environment. But maybe someone will want to modify the response header, who knows?