Investigate why Sequencing Server Tests are Flaky

[Mythicaeda]

Title is an apt description. The Sequencing Server Tests have been flaky as of late. Completion of this ticket involves investigating why and resolving it.

Example Test Runs where this occurred: https://github.com/NASA-AMMOS/aerie/actions/runs/13063337782/attempts/1 https://github.com/NASA-AMMOS/aerie/actions/runs/13035929860/attempts/1

[Mythicaeda]

Update: on further investigation today, I noticed that the flakiness tends to be due to the tests losing connection with the Gateway partway through (url: http://localhost:9000/).

Example HTML/XML test report from today: test-report.html test-report.xml

The errors progress as follows:

FetchError: request to http://localhost:9000/file failed, reason: write EPIPE
FetchError: request to http://localhost:9000/auth/login failed, reason: socket hang up
FetchError: request to http://localhost:9000/auth/login failed, reason: read ECONNRESET
FetchError: request to http://localhost:9000/auth/login failed, reason: connect ECONNREFUSED ::1:9000

Assuming the tests are output in execution order, then eventually the connection issue is resolved and the tests begin to pass again.

So, it looks like resolving this EPIPE error may fix the flakiness. One thing worth checking is whether this is something only the /file endpoint can cause.