Add a Shapeless project test

[jleider]

trafficstars

I understand that Shapeless is not part of the core Scala library and this request may be out of scope but as a very popular generic programming lib for Scala it would be nice to see if the scala compiler was improving its performance on Shapeless related projects. For all its utility, code depending on Shapeless tends to be very slow and it would be great to get a visual on the performance of this vital library.

I would be more than happy to provide a simple shapeless test if this suggestion gets any traction.

[milessabin]

I'm going to be doing something along exactly these lines over the next few weeks. If you have particular usage patterns which are causing you compile time or runtime performance problems I'd be super keen to see them :-)

[beezee]

@milessabin that's great to hear. I work with @jleider and I'd love to share some of our slower-to-compile uses (haven't noticed any runtime issues at all.)

Generally we have seen typeclass derivation, Merger, and Align to all have dramatic impact when called for around the codebase.

We also have observed slow compilation on a file which hosts a large Coproduct (~20 members) and several Poly1 definitions over it. Interesting note on one of the Poly1 definitions, it uses a typeclass for which we have a catch all definition over a structural type, and a small handful of more specific instances defined in higher priority scope. Here's a contrived illustration in case I didn't explain well:

trait CanDoThing[A] {
  def doThing(a: A): Int
}

type IsDoThing = {
   def getInt: Int
}

// this is low priority catchall
implicit def canDoThing[A <: IsDoThing] = new CanDoThing[A] {
  def doThing(a: A) = a.getInt
}

// then some more specific instances for non-conforming types
implicit val specificDoThing = ???

type DoThingCP = A :+: B :+: C :+: etc :+: CNil

object DoThingPoly extends Poly1 {
  implicit def caseCanDoThing[A)(implicit cd: CanDoThing[A]) = at[A](a => cd.doThing(a) + 1)
}

The other Poly defs are straightforward and explicitly address each member of the Coproduct.

[milessabin]

Have you tried Typelevel Scala with the -Yinduction-heuristics flag enabled, and if so has it improved these scenarios at all?

Nb. currently use of Lazy will prevent these heuristics from being applied (that's something I'm currently working on fixing), but it should help with Align and Merger (which don't use Lazy).

[jleider]

@milessabin Benchmarking our slowest project (91 files) the -Yinduction-heuristics flag with Scala 2.11.11-typelevel-4 shows a ~25s improvement on a 300s compile. Clearly there is something else at work here, perhaps its the typeclasses.

[jvican]

@jleider Have a special look at macro expansions, if you can.