scoobi icon indicating copy to clipboard operation
scoobi copied to clipboard
verified publisher icon NICTA

Add retry on hadoop client calls

Open raronson opened this issue 11 years ago • 0 comments

If the cluster is under a lot of load, client calls which directly interact with it can fail in weird ways. I have seen LeaseExpiredException, ConnectException: Connection refused, and others which all point to a cluster under load not able to respond to requests.

Scoobi should retry these client calls a number of times so the app doesn't die on the first exception from a hadoop call.

Examples:

  • [ ] Counters
  • [ ] HDFS interaction
  • [ ] Other?

raronson avatar Sep 16 '14 07:09 raronson