aibrix The current e2e tests are flaky

🐛 Describe the bug

Steps to Reproduce

No response

Expected behavior

No response

Environment

No response

Feb 12 '25 01:02 Jeffwan

this is kind of painful now. the job takes very long, eventually failed and it lots of runtime.

Feb 13 '25 17:02 Jeffwan

@varungup90 If you have not started this issue, I will take it over.

Feb 13 '25 17:02 Jeffwan

I can reproduce the issue on my end but my setup is not 100% same as kind setups. I run e2e against my mac for desktop directly. I will double check that part and come back to this issue

Feb 13 '25 19:02 Jeffwan

What is the rootcause?

Feb 13 '25 21:02 varungup90

unexpected EOF is the last issue. It happens to miss one request.

Feb 14 '25 01:02 Jeffwan

https://github.com/openai/openai-go?tab=readme-ov-file#retries

It actually retried up to 3 times for 500 like issues. However, seems last request is commonly lost, ends up seeing 401 unexpected EOF issues. It has been verified by checking the headers for coming requests

This could be gateway extension problems. I do not know the root cause. One thing we can do is to disable retry for this case.

client := openai.NewClient(
	option.WithMaxRetries(0), // default is 2
)

Feb 14 '25 01:02 Jeffwan

Presently, e2e tests are working fine. I will close this issue.

Apr 04 '25 17:04 varungup90