What is the best way to handle errors?

[kdonthi]

I was initially using panics to handle errors because it seemed like the FailedTasks supports using panics.

It seems a little weird to see a panic every time an error happens though; what is the canonical/right way to handle the error?

[alitto]

Hey @kdonthi!

Yes, panics are probably not the best way to handle all application errors, but I think it's important to distinguish between two general kinds of errors:

• Recoverable errors: These are transient errors that may resolve themselves over time or are related to client-side issues. Concrete examples of this could include a database query timing out due to resource exhaustion or a client submitting an HTTP request with an invalid payload.
• Non-recoverable errors: These are errors that the program will never be able to recover from on its own. A concrete example of this would be an invalid database hostname being passed to the program via environment variables, or a YAML configuration file used by the program containing invalid YAML syntax.

As a rule of thumb, you would want to panic only for non-recoverable errors and use regular error objects for recoverable errors. Continuing with the invalid database hostname example, a program can't determine the correct hostname to access the database on its own, so there's no point in continuing its execution, and this is where panics are helpful.

Now, in the context of this particular library, worker pools do not provide any mechanism to handle "recoverable errors" thrown inside tasks submitted to them, they only provide a way to configure a custom panic handler to perform some actions in the event of an unexpected non-recoverable error is thrown in one of the tasks. This is an intentional design decision aimed at simplyfing the task interface used by Submit methods. There is a wide range of possible interfaces for tasks (e.g. returning an error, returning a task result object + error, etc) we went for the simpler one and assume each task knows how to handle recoverable errors internally :slightly_smiling_face:.

Here's a simple example on a task that sends all recoverable errors to a channel to log them:

// Create a pool with 10 workers and a buffer of 1000 tasks
pool := pond.New(10, 1000)

errors := make(chan error, 0)

// Submit one or more tasks that can fail with recoverable errors
pool.Submit(func() {
    err := doSomething()
    if err != nil {
        errors <- err
    }
})

// Wait until all tasks have completed
pool.StopAndWait()

// Read all errors
for err := range errors {
    fmt.Println(err)
}