etl
etl copied to clipboard
Some tcpinfo rows too large.
Inserts are sometimes failing on tcpinfo buffers. Likely due to large number of snapshots for some rows.
Should make two changes:
- Limit number of snapshots in a row. Perhaps filter down to 10/second after the first 12 seconds, and 1/second after the first 60 seconds?
- Handle insert errors by submitting rows individually.
#2 is done. The buffer is now split in half and retried. Only single row failures are not retried.