Sayma DRTIO breaks, GTH/Ultrascale/Vivado flakey

[jbqubit]

In https://github.com/m-labs/artiq/issues/795#issuecomment-605371138 @sbourdeauducq said

Sometimes (and rarely AFAICT) the DRTIO link appear to become corrupted with some bitstream builds (visible symptom is a storm of broken aux packets); making a minor/unrelated change to the gateware code appears to "fix" it. I have never seen this on Kasli so this is probably GTH/Ultrascale specific.

I suppose Xilinx tech support might be able to shed some light into this. Do you have examples suitable for submission to Xilinx illustrating this behavior?

[sbourdeauducq]

It is very hard to isolate the problem. Just changing the version string (i.e. a completely unrelated RAM initialization value) is sufficient to cause the issue or make it disappear. All I could send is a full ARTIQ bitstream that exhibits broken DRTIO. I don't know what Xilinx can make out of something like that.

[sbourdeauducq]

Could be the same as https://github.com/m-labs/artiq/issues/1424

[HarryMakes]

I recently replicated this DRTIO breakage on Sayma with a piece of modified ARTIQ gateware. Such modification involved replacing the two TTL InOuts at the front-panel MCX connectors with a new TTL SERDES output driver that is hardcoded with a 9 MHz square wave, decimated at the 4 times the RTIO frequency (150 MHz * 4 = 600 MHz). When the square wave is at 9 MHz, no DRTIO breakage is observed with repeated power-cycling or FPGA reloads. However, when I halve the frequency to 4.5 MHz, DRTIO breakage usually doesn't occur at the first FPGA reload, but does occur on subsequent power-cycling.

Since https://github.com/m-labs/misoc/pull/108 should have fixed the Ethernet breakage on GTH, I am going to test the same fix on DRTIO's GTHs as well, and see if this resolves the issue.

cc: @jbqubit