How to debug the compilation error "Cannot extract value from singleton.ops.impl.OpMacro"?

[tribbloid]

Hi @fthomas @soronpo Thanks a lot for the great work, I'm one of the supporting developer of the deep learning library kotlingrad. When we try to integrate singleton-ops for the project on some tensor operations, we encounter some strange compiler behaviours that are hard to isolate or simplify, so we have to post a link to our repository in case others want to reproduce, sorry.

In one of our test cases: https://github.com/tribbloid/shapesafe/blob/59405baedc9048a9cac1fbf2bb7708c917266a6e/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala#L167

The compiler / macro misbehave on 2 apparently equivalent code blocks:

      val v0 = DoubleVector.random(6)
      val v1 = v0.pad(3)

      {
        val result = v1.reify

        val aa = result.arity

        {
          print_@(WideTyped(result.arity).viz)
          // this works
          result.arity.internal.dummyImp(3)
          result.crossValidate()
          result.arity.internal.requireEqual(12)
        }

        {
          print_@(WideTyped(aa).viz)
          // this doesn't, how did it happened?
          aa.internal.dummyImp(3)
          aa.internal.requireEqual(12)
        }
      }

These are the compilation errors:

[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:188: implicit error;
!I proof (Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))): OpMacro[+, S, Int, Int]
[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:189: implicit error;
!I proof (Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.Require,singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.==,S,Int(12),Int(0)],String("Cannot prove requirement Require[...]"),singleton.ops.impl.NoSym]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.Require, List()), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$eq$eq, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(12)), FoldableConstantType(Constant(0)))), FoldableConstantType(Constant(Cannot prove requirement Require[...])), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.NoSym, List())))): OpMacro[Require, OpMacro[==, S, Int, Int], String, NoSym]
two errors found

> Task :core:compileTestScala FAILED

These are the same error if the implicit debugging plugin "splain" (https://github.com/tek/splain) is disabled:

[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:188: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:189: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.Require,singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.==,S,Int(12),Int(0)],String("Cannot prove requirement Require[...]"),singleton.ops.impl.NoSym]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.Require, List()), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$eq$eq, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(12)), FoldableConstantType(Constant(0)))), FoldableConstantType(Constant(Cannot prove requirement Require[...])), TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.NoSym, List())))
two errors found

> Task :core:compileTestScala FAILED

My questions are:

1. What's the cause of this error? It clearly won't be caused by a missing dependent type (variable aa has a static path, but result.arity doesn't) So how is the macro unable to figure out the type for aa?

2. What should we do for any error that has such message? Should we debug or try to print the stacktrace?

[tribbloid]

The full context of this error is displayed in the CI output:

https://github.com/tribbloid/shapesafe/runs/925426505?check_suite_focus=true#step:4:76

[soronpo]

From what I see in the error you have an operation S + 3, but S is unknown. So you are loosing refinement somewhere along the way. Maybe it's possible to improve the error messages.

[tribbloid]

That's the obvious part, the no-so-obvious part is why result.arity has the refinement correctly.

Also, the S should refer to result.arity.S theoretically, let me try to infer its full type hierarchy using reflection ...

[tribbloid]

The result.arity.S can't generate a TypeTag automatically, so I print the type hierarchy of result.arity instead:

edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[Int(6)]#Out#Out
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[Int(6)]
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const[Int(6)]
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Invar[Int(6)]
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Out_=[edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const[Int(6)]]
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Arity
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Proven
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Proof
	- java.io.Serializable
	- edu.umontreal.kotlingrad.shapesafe.m.arity.Operand
	- Object
	- Any

The type definition on the root level is Arity.FromLiteral[S] which means S resolves to Witness narrow type Int(6), I hope this is refined enough for you?

[soronpo]

Can you please do the following:

1. clone the singleton-ops library.
2. flip the VerboseTravel flag https://github.com/fthomas/singleton-ops/blob/master/src/main/scala/singleton/ops/impl/GeneralMacros.scala#L342
3. publish it locally.
4. configure your project to rely on the locally publised singleton-ops.
5. recompile your example and paste the compilation log here (first have a successful compilation, and then uncomment the problematic code and get just incremental compilation log).

Afterwords, go back to the original configuration. The log is huge and will slow you down.

[tribbloid]

Thanks a lot, here is the relevant part of the log (logging level set to --info):

@@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
-->
  @@OpCalc@@
  TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
  RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
  @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
  TP: S
  RAW: TypeRef(NoPrefix, TypeName("S"), List())
---->
    @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
    TP: 
    RAW: TypeBounds(TypeRef(ThisType(scala), scala.Nothing, List()), TypeRef(ThisType(scala), scala.Any, List()))
------>
      @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
      TP: Any
      RAW: TypeRef(ThisType(scala), scala.Any, List())
-------->
        @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
        TP:  {
          final def ==(x$1: Any): Boolean
          final def !=(x$1: Any): Boolean
          def equals(x$1: Any): Boolean
          def hashCode(): Int
          def toString(): String
          def getClass(): Class[_]
          final def isInstanceOf[T0]: Boolean
          final def asInstanceOf[T0]: T0
          final def ## : Int
        }
        RAW: ClassInfoType(List(), Scope(TermName("$eq$eq"), TermName("$bang$eq"), TermName("equals"), TermName("hashCode"), TermName("toString"), TermName("getClass"), TermName("isInstanceOf"), TermName("asInstanceOf"), TermName("$hash$hash")), scala.Any)
---------->
          Exhausted search at: ClassInfoType(List(), Scope(TermName("$eq$eq"), TermName("$bang$eq"), TermName("equals"), TermName("hashCode"), TermName("toString"), TermName("getClass"), TermName("isInstanceOf"), TermName("asInstanceOf"), TermName("$hash$hash")), scala.Any)
<----------
<--------
<------
<----
  @@Unknown@@
  TP: S
  RAW: TypeRef(NoPrefix, TypeName("S"), List())
  @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
  TP: Int(3)
  RAW: FoldableConstantType(Constant(3))
---->
<----
  Exhausted search at: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
<--
@@Unknown@@
TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
!!!!!!aborted with: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0)))) at Some(trait OpMacro), Some(trait OpMacro)
@@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
-->
  @@OpCalc@@
  TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
  RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
  @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
  TP: S
  RAW: TypeRef(NoPrefix, TypeName("S"), List())
---->
    @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
    TP: 
    RAW: TypeBounds(TypeRef(ThisType(scala), scala.Nothing, List()), TypeRef(ThisType(scala), scala.Any, List()))
------>
      @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
      TP: Any
      RAW: TypeRef(ThisType(scala), scala.Any, List())
-------->
        @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
        TP:  {
          final def ==(x$1: Any): Boolean
          final def !=(x$1: Any): Boolean
          def equals(x$1: Any): Boolean
          def hashCode(): Int
          def toString(): String
          def getClass(): Class[_]
          final def isInstanceOf[T0]: Boolean
          final def asInstanceOf[T0]: T0
          final def ## : Int
        }
        RAW: ClassInfoType(List(), Scope(TermName("$eq$eq"), TermName("$bang$eq"), TermName("equals"), TermName("hashCode"), TermName("toString"), TermName("getClass"), TermName("isInstanceOf"), TermName("asInstanceOf"), TermName("$hash$hash")), scala.Any)
---------->
          Exhausted search at: ClassInfoType(List(), Scope(TermName("$eq$eq"), TermName("$bang$eq"), TermName("equals"), TermName("hashCode"), TermName("toString"), TermName("getClass"), TermName("isInstanceOf"), TermName("asInstanceOf"), TermName("$hash$hash")), scala.Any)
<----------
<--------
<------
<----
  @@Unknown@@
  TP: S
  RAW: TypeRef(NoPrefix, TypeName("S"), List())
  @@TypeCalc.unapply@@ source-/home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala,line-185,offset=4376
  TP: Int(3)
  RAW: FoldableConstantType(Constant(3))
---->
<----
  Exhausted search at: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
<--
@@Unknown@@
TP: singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
RAW: TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))
!!!!!!aborted with: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0)))) at Some(trait OpMacro), Some(trait OpMacro)
[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:185: implicit error;
!I proof (Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,S,Int(3),Int(0)]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(NoPrefix, TypeName("S"), List()), FoldableConstantType(Constant(3)), FoldableConstantType(Constant(0))))): OpMacro[+, S, Int, Int]

Hope it could be useful

[tribbloid]

@soronpo Thanks a lot for your attention, did you observe anything strange in the log?

[soronpo]

Thanks for the reminder. Forgot to respond. From the looks of it, indeed it's not singleton-ops territory. From the getgo the S is fed as an argument and there is nothing we can do about it. So the refinement is lost at the use-site. Maybe running the compiler with -Vtyper would yield some more information for you to understand where the refinement is lost.

[tribbloid]

It looks like the Vtyper feature is highly experimental: I got the following error on a very ordinary generic case class:

caught scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving method copy: while typing new VizType

[soronpo]

It looks like the Vtyper feature is highly experimental: I got the following error on a very ordinary generic case class:

caught scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving method copy: while typing new VizType

Try with -Ytyper-debug instead

[tribbloid]

-Ytyper-debug gave almost identical error. The full scope of the error looks like this:

(I guess this is more suitable to be posted on scalabug issue tracker, bu here you go)

|    |    |    |    |-- new VizType(tpe) EXPRmode (site: method copy in VizType) 
|    |    |    |    |    |-- new VizType APPSELmode-BYVALmode-EXPRmode-FUNmode-POLYmode (silent: method copy in VizType) 
|    |    |    |    |    |    |-- new VizType APPSELmode-EXPRmode-POLYmode-QUALmode (silent: method copy in VizType) 
|    |    |    |    |    |    |    |    caught scala.reflect.internal.Symbols$CyclicReference: illegal cyclic reference involving method copy: while typing new VizType
## Exception when compiling 7 sources to /home/peng/git/shapesafe/graph-commons/build/classes/scala/main
java.lang.AssertionError: assertion failed: (VizType,case class WideTyped[T] extends scala.Product with scala.Serializable {
  <caseaccessor> <paramaccessor> private[this] val value: T = _;
  def <init>(value: T): graph.commons.util.WideTyped[T] = {
    super.<init>();
    ()
  };
  type Wide = T;
  def viz(implicit ttag: TypeTag[T]): graph.commons.util.viz.VizType = VizType[T]
})
scala.reflect.internal.SymbolTable.throwAssertionError(SymbolTable.scala:170)
scala.tools.nsc.typechecker.TypersTracking$typingStack$.pop(TypersTracking.scala:98)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6026)
scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:6051)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedStats$8(Typers.scala:3397)
scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3397)
scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:5582)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5885)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5973)
scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.apply(Analyzer.scala:117)
scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:454)
scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.run(Analyzer.scala:105)
scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1515)
scala.tools.nsc.Global$Run.compileUnits(Global.scala:1499)
scala.tools.nsc.Global$Run.compileSources(Global.scala:1491)
scala.tools.nsc.Global$Run.compile(Global.scala:1626)
xsbt.CachedCompiler0.run(CompilerInterface.scala:153)
xsbt.CachedCompiler0.run(CompilerInterface.scala:125)
xsbt.CompilerInterface.run(CompilerInterface.scala:39)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
sbt.internal.inc.AnalyzingCompiler.call(AnalyzingCompiler.scala:248)
sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:122)
sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:95)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4(MixedAnalyzingCompiler.scala:91)
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:186)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$3(MixedAnalyzingCompiler.scala:82)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$3$adapted(MixedAnalyzingCompiler.scala:77)
sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:215)
sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:77)
sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:146)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:343)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:343)
sbt.internal.inc.Incremental$.doCompile(Incremental.scala:120)
sbt.internal.inc.Incremental$.$anonfun$compile$4(Incremental.scala:100)
sbt.internal.inc.IncrementalCommon.recompileClasses(IncrementalCommon.scala:180)
sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:98)
sbt.internal.inc.Incremental$.$anonfun$compile$3(Incremental.scala:102)
sbt.internal.inc.Incremental$.manageClassfiles(Incremental.scala:155)
sbt.internal.inc.Incremental$.compile(Incremental.scala:92)
sbt.internal.inc.IncrementalCompile$.apply(Compile.scala:75)
sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:348)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:301)
sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:168)
sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:248)
sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:74)
org.gradle.api.internal.tasks.scala.ZincScalaCompiler.execute(ZincScalaCompiler.java:147)
org.gradle.api.internal.tasks.scala.ZincScalaCompilerFacade.execute(ZincScalaCompilerFacade.java:47)
org.gradle.api.internal.tasks.scala.ZincScalaCompilerFacade.execute(ZincScalaCompilerFacade.java:31)
org.gradle.api.internal.tasks.compile.daemon.AbstractDaemonCompiler$CompilerWorkAction.execute(AbstractDaemonCompiler.java:135)
org.gradle.workers.internal.DefaultWorkerServer.execute(DefaultWorkerServer.java:63)
org.gradle.workers.internal.AbstractClassLoaderWorker$1.create(AbstractClassLoaderWorker.java:49)
org.gradle.workers.internal.AbstractClassLoaderWorker$1.create(AbstractClassLoaderWorker.java:43)
org.gradle.internal.classloader.ClassLoaderUtils.executeInClassloader(ClassLoaderUtils.java:97)
org.gradle.workers.internal.AbstractClassLoaderWorker.executeInClassLoader(AbstractClassLoaderWorker.java:43)
org.gradle.workers.internal.IsolatedClassloaderWorker.run(IsolatedClassloaderWorker.java:49)
org.gradle.workers.internal.IsolatedClassloaderWorker.run(IsolatedClassloaderWorker.java:30)
org.gradle.workers.internal.WorkerDaemonServer.run(WorkerDaemonServer.java:85)
org.gradle.workers.internal.WorkerDaemonServer.run(WorkerDaemonServer.java:55)
org.gradle.process.internal.worker.request.WorkerAction$1.call(WorkerAction.java:138)
org.gradle.process.internal.worker.child.WorkerLogEventListener.withWorkerLoggingProtocol(WorkerLogEventListener.java:41)
org.gradle.process.internal.worker.request.WorkerAction.run(WorkerAction.java:135)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
java.lang.Thread.run(Thread.java:748)
           

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':graph-commons:compileScala'.
> assertion failed: (VizType,case class WideTyped[T] extends scala.Product with scala.Serializable {
    <caseaccessor> <paramaccessor> private[this] val value: T = _;
    def <init>(value: T): graph.commons.util.WideTyped[T] = {
      super.<init>();
      ()
    };
    type Wide = T;
    def viz(implicit ttag: TypeTag[T]): graph.commons.util.viz.VizType = VizType[T]
  })

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 14s

[soronpo]

Maybe it's a compiler flag. Maybe some instability as some improper macro use. TypeTag stuff maybe make things problematic.

[tribbloid]

agreed, after spending some time playing with the compiler it appears that all implicit inference from symbols like S has the same problem, if I replace the line that causes the error with:

universe.typeTag[aa.SS]

I got a similar error:

[Error] /home/peng/git/shapesafe/core/src/test/scala/edu/umontreal/kotlingrad/shapesafe/core/tensor/DoubleVectorSpec.scala:197: implicit error;
!I ttag (No TypeTag available for Invar.this.SS): TypeTag[S]

This in the end can be traced back to a alleged compiler bug:

https://stackoverflow.com/questions/59708880/in-scala-why-it-is-impossible-to-infer-typetag-from-type-alias-or-dependent-typ

Of which the cause is still inconclusive. Let me try some other tricks before figuring out a way to bypass it

[tribbloid]

No it was not relevant, original ticket (https://github.com/scala/scala/pull/5836) merely shows a feature being removed due to conflict with other behaviour of the compiler.

So I end up debugging it in runtime using reflection:

          import ScalaReflection.universe

          def sniff[T](v: T)(implicit ttag: universe.TypeTag[T]) = ttag

          val aaT = sniff(aa).tpe
          print(universe.showRaw(aaT.typeSymbol.typeSignature))

Which gives the proper type argument you have observed:

PolyType(List(TypeName("S")), ClassInfoType(List(TypeRef(ThisType(scala), TypeName("AnyRef"), List()), TypeRef(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const, List(TypeRef(NoPrefix, TypeName("S"), List())))), Scope(TermName("singleton"), TermName("singleton "), termNames.CONSTRUCTOR, TermName("number")), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral))

But it is before the resolution! Using the baseType of the type directly (instead of the type signature) reveals the true nature of the type argument:

          val sT = aaT.baseType(aaT.baseClasses.head).typeArgs.head

          print(universe.showRaw(sT))

gives the refined type:

FoldableConstantType(Constant(6))

Of course, I was conducting the experiment in runtime universe, and the compile-time may follow an entirely different path. Regardless, do you think the above baseType trick can be used in your code? Do you have a test class responsible for OpMacro summoning on edge cases like this? I'll see if I can compile it into shorter and more reproducible code test cases. Hopefully we can get a patch ready ASAP.

[soronpo]

Can you post a print of the aaT.typeSymbol.typeSignature without the showRaw?

[tribbloid]

yes sir, println(aaT.typeSymbol.typeSignature) yields:

[S <: Int]AnyRef
        with edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const[S] {
  val singleton: S
  private[this] val singleton: S
  def <init>(singleton: S): edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[S]
  override def number: Int
}

BTW println(sT) yields:

Int(6)

[soronpo]

If you will be able to minimize this, it would help a lot. Can you please provide the following:

println(aaT)
println(universe.showRaw(aaT))
println(aaT.dealias)
println(universe.showRaw(aaT.dealias))
println(aaT.typeSymbol)
println(universe.showRaw(aaT.typeSymbol))
println(aaT.baseType)
println(universe.showRaw(aaT.baseType))

[tribbloid]

At your service:

          println(aaT)

edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[Int(6)]#Out#Out

          println(universe.showRaw(aaT))

TypeRef(TypeRef(TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral, List(FoldableConstantType(Constant(6)))), TypeName("Out"), List()), TypeName("Out"), List())

          println(aaT.dealias)

edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[Int(6)]

          println(universe.showRaw(aaT.dealias))

TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral, List(FoldableConstantType(Constant(6))))

          println(aaT.typeSymbol)

class FromLiteral

          println(universe.showRaw(aaT.typeSymbol))

edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral

          println(
            aaT.baseClasses.map { clazz =>
              val baseType = aaT.baseType(clazz)
              baseType
            }
          )

List(edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral[Int(6)], edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const[Int(6)], edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Invar[Int(6)], edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Out_=[edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const[Int(6)]], edu.umontreal.kotlingrad.shapesafe.m.arity.Arity, edu.umontreal.kotlingrad.shapesafe.m.arity.Proven, edu.umontreal.kotlingrad.shapesafe.m.arity.Proof, java.io.Serializable, edu.umontreal.kotlingrad.shapesafe.m.arity.Operand, Object, Any)

          println(
            aaT.baseClasses.map { clazz =>
              val baseType = aaT.baseType(clazz)
              universe.showRaw(baseType)
            }
          )

List(TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.FromLiteral, List(FoldableConstantType(Constant(6)))), TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const, List(FoldableConstantType(Constant(6)))), TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Proof), edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Invar, List(FoldableConstantType(Constant(6)))), TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Proof), edu.umontreal.kotlingrad.shapesafe.m.arity.Proof.Out_$eq, List(TypeRef(SingleType(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity.Const, List(FoldableConstantType(Constant(6)))))), TypeRef(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Arity, List()), TypeRef(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Proven, List()), TypeRef(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Proof, List()), TypeRef(ThisType(java.io), java.io.Serializable, List()), TypeRef(ThisType(edu.umontreal.kotlingrad.shapesafe.m.arity), edu.umontreal.kotlingrad.shapesafe.m.arity.Operand, List()), TypeRef(ThisType(java.lang), java.lang.Object, List()), TypeRef(ThisType(scala), scala.Any, List()))

The following 2 phrases doesn't compile:

println(aaT.baseType)
println(universe.showRaw(aaT.baseType))

so I twisted them a bit.

[tribbloid]

the result is also displayed in the CI: https://github.com/tribbloid/shapesafe/runs/981704487?check_suite_focus=true#step:4:281

[soronpo]

The 6 type value isn't making its way into singleton-ops, so it looks like the typetag is inspecting the wrong thing. For example. If FromLiteral was defined like so:

trait FromtLiteral[T] {
  type Out = T
}
val fl : FromLiteral[6] = ???

Then you implicitly[fl.Out + 3] would have worked for sure.

[tribbloid]

@soronpo sorry for the slow reply, I just tried your example and it reproduces the error:

object ErrorCase0 {

  case class Outer[S]() {

    type SS = S
  }

  val f1 = Outer[Witness.`6`]()

  implicitly[f1.SS + Witness.`3`]
}

if compiled with either scala 2.12.11 or 2.13.3, the compile reports:

[Error] /home/peng/git-spike/scalaspike/common/src/test/scala/com/tribbloids/spike/singleton_ops_spike/ErrorCase0.scala:15: Cannot extract value from singleton.ops.impl.OpMacro[singleton.ops.impl.OpId.+,Any,Any,singleton.ops.NP]
showRaw==> TypeRef(ThisType(singleton.ops.impl), singleton.ops.impl.OpMacro, List(TypeRef(SingleType(ThisType(singleton.ops.impl), singleton.ops.impl.OpId), singleton.ops.impl.OpId.$plus, List()), TypeRef(ThisType(scala), scala.Any, List()), TypeRef(ThisType(scala), scala.Any, List()), TypeRef(SingleType(SingleType(SingleType(ThisType(<root>), singleton), singleton.ops), singleton.ops.package), TypeName("NP"), List())))

[tribbloid]

If you like to play with it, voila:

https://scastie.scala-lang.org/tribbloid/3Ijg2dSLRcmiiNrXnSf4hg

[soronpo]

You need W.`XXX`.T https://scastie.scala-lang.org/pL4vfrNISziWcKHUBlX8fA

[tribbloid]

@soronpo I'm so sorry, after 2 weeks I no longer remember how shapeless works :facepalm:

I think I've finally isolate the problem, all caused by this.type as an abstract type:

import shapeless.Witness
import singleton.ops.+

import scala.language.implicitConversions

object ErrorCase0 {

  trait Proof extends Serializable {

    type Out <: Arity
    def out: Out
  }

  trait Arity extends Proof {

    override type Out = this.type
    override def out: Out = this
  }

  object Arity {

    trait Const[S] extends Arity with Proof {

      type SS = S
    }

    case class FromLiteral[S <: Int]() extends Const[S] {}
  }

  val v1 = Arity.FromLiteral[Witness.`6`.T]()
//  val f1 = Arity(6)
  implicitly[v1.SS + Witness.`3`.T]

  val v2 = v1.out
  implicitly[v2.SS + Witness.`3`.T] // this fails
}

https://scastie.scala-lang.org/tribbloid/yRXvkpWWRiWAyGHzpI9g1Q/1

Scalac was known for fumbling on this.type inference. This issue may be one instance of such fumbling and may won't be fixed without some major compiler overhaul. Regardless, the above case may worth further investigation to be 100% sure of the cause

For the moment I'll just replace the abstract type with an F-bounded type parameter to circumvent it. Thank you so much for your instructions!