pathway icon indicating copy to clipboard operation
pathway copied to clipboard

[Bug]: csv connector should retry, not fail

Open izulin opened this issue 8 months ago • 0 comments

Steps to reproduce

csv connector under faulty network fails, instead of retrying

Relevant log output

E0614 11:16:48.882455 140007979890240 mod.rs:311] There had been an error processing the row read result: CSV error: record 550622 (line: 550623, byte: 76923319): found record with 1 fields, but the previous record has 9 fields
E0614 11:16:48.925295 140007979890240 mod.rs:311] There had been an error processing the row read result: failed to perform S3 operation GetObject reason: Io(Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" })
thread 'pathway:send-3' panicked at external/timely-dataflow/communication/src/allocator/zero_copy/bytes_exchange.rs:96:48:
MergeQueue poisoned.

What did you expect to happen?

retry reading data

Version

0.13.0

Docker Versions (if used)

No response

OS

Linux

On which CPU architecture did you run Pathway?

None

izulin avatar Jun 14 '24 09:06 izulin