apache
Initialize AuTest port queue concurrently
On WSL, this change reduces the runtime for the AuTest setup from prohibitive to annoying. Most developers are not using WSL and could possibly see a performance degradation with this change.
Opening a connection to a host port is a very slow operation on WSL: 0.5s * 1000 ports = 500s opening ports, not accounting for other overhead. In my PoC script, it took 20 seconds after this change - more than I hoped. I think this is due to not being able to set a 0.5 second timeout with the async socket interface like we do with the synchronous one. The actual change in this patch takes more than 20 seconds but is still sub-minute on my system.
This needs testing to see how it performs in typical dev environments before it can be considered for merging.
Chiming in late on this.. The goal is to secure all the necessary ports. ie, enough for a given test and enough buffer for ports to reset correctly with the system for various reasons. This is the main goal we have for getting the port set as we do.
The speed concern is that we are getting each port serially. By "getting a port" I mean testing that the port is "safe". The goal here I believe is that you did testing to see that having this do an async for loop to get the ports was faster.
Is that correct?
@dragon512 Thank you for asking for clarification. Your understanding of my concern is (mostly? see note) correct, but to avoid getting tunnel visioned it might be better to state the concern as "connecting to a Windows host port from WSL through the Python socket API to see if it is open has too high a runtime cost for our serial port searching algorithm."
In this PR, Brian and I have worked on trying to change the algorithm. However, other approaches might be better; for example, not using WSL to run AuTests. I could also try to understand why the runtime cost is so large and possibly try to fix the issue in WSL itself. I didn't want to take that on initially, because it would probably be a great deal of work and require some WSL knowledge and expertise.
Edit: One note. I am not sure whether you mean an async for loop. There is an async for construct in Python, as you probably know, which we haven't used. We are spawning an asynchronous task to check each port without blocking until a task completes. While waiting for the IO for one port, we want to do IO for another port. An async for loop does not run the elements of its iterable concurrently and thus doesn't accomplish this goal.