Inconsistent `ApplicativeError` behaviour for `OptionT[F[_], A]` / `EitherT[F[_], A, B]` if `F[_]` also has an `ApplicativeError` instance

[yangzai]

For example, if F[_] is Either[Throwable, *], there would exist 2 instances of ApplicativeError for OptionT[Either[Throwable, *], *]:

ApplicativeError[OptionT[Either[Throwable, *], *], Throwable] // higher priority instance derived from `F[_]`
ApplicativeError[OptionT[Either[Throwable, *], *], Unit] // default instance for all `OptionT`

The instance derived from F[_] would be higher in priority and this would result in inconsistency with the default behaviour (when F[_] does not have an ApplicativeError instance), e.g.:

import cats.implicits._
import cats.{Applicative, ApplicativeError, Monoid}
import cats.data.OptionT

def handleErrorWithMonoidEmpty[F[_], A, E: ApplicativeError[F, *]](fa: F[A])(implicit A: Monoid[A]) = fa.handleError(_ => A.empty)

// if `F[_]` is `List`, it handles `None` as error
handleErrorWithMonoidEmpty(OptionT.none[List, Int]) //OptionT(List(Some(0)))

// if `F[_]` is `Either[Throwable, *]`, it handles for `Throwable` instead
handleErrorWithMonoidEmpty(OptionT.none[Either[Throwable, *], Int]) //OptionT(Right(None))
handleErrorWithMonoidEmpty(new Throwable("throw").raiseError[OptionT[Either[Throwable, *], *], Int]) //OptionT(Right(Some(0)))

~~The same issue exist for EitherT:~~ The instance priorities are reversed for EitherT:

ApplicativeError[EitherT[Either[Throwable, *], String, *], Throwable]
ApplicativeError[EitherT[Either[Throwable, *], String, *], String] //this would be higher in priority

[djspiewak]

This is basically by design, though it's fair to argue whether or not it's intuitive or correct. When you nest EitherTs, you will, by definition, get several MonadError instances out of it. Cats prioritizes these so that it doesn't cause ambiguity, but it can feel a bit arbitrary at times.

[joroKr21]

But if you use ApplicativeError[EitherT[Either[Throwable, *], Throwable, *], Throwable] it will derive the outer one, not the inner one so that has higher priority I believe.

[yangzai]

@joroKr21 You are right, the priorities are reversed for EitherT, which is still confusing that they are different.

@djspiewak In my case I was looking for a neat way to handle both errors but in @joroKr21 's example where the error types are the same there would be no way to explicitly summon the alternative instance either. Maybe in Scala 3 something like ApplicativeError[OptionT[Either[Throwable, *], *], Throwable | Unit] could work.

I will close this if there's nothing to be done. We could reprioritise the instances I suppose, but I agree that correctness in this case can be subjective.

[djspiewak]

IMO as long as it's unambiguous, it's okay, but I think we can hold off on closing it until we hear others' opinions.