deltacat icon indicating copy to clipboard operation
deltacat copied to clipboard
verified publisher icon ray-project

Leverage Daft Transient Errors to retry tasks on ray and drop timeout times

Open samster25 opened this issue 1 year ago • 3 comments

  • Upgrade Daft to 0.2.23
  • Cut down connect timeout to 5 seconds and read timeout to 10 seconds
  • Add Daft Transient error to retry list for ray
  • This version of daft also will not panic on connection issues.

samster25 avatar May 02 '24 19:05 samster25

This leverages: https://github.com/Eventual-Inc/Daft/pull/2197 and https://github.com/Eventual-Inc/Daft/pull/2214

samster25 avatar May 02 '24 19:05 samster25

We are upgrading Ray version. We will plan this deployment after observing new Ray version to isolate issues that may be related to either of the changes.

raghumdani avatar May 04 '24 03:05 raghumdani

Thanks for this PR. Are connections per file still relevant?

Still should be, the number of connections per file let's you tune between throughput and number of failures before we retry.

samster25 avatar May 06 '24 08:05 samster25