Macros missing for Scala 3

[jmcclell]

The following code compiles and works as expected on Scala 2.13.x, but fails on 3.0.0-RC1:

import eu.timepit.refined.numeric._
import eu.timepit.refined.api.Refined
import eu.timepit.refined.auto._

val x: Int Refined Positive = 5

[error] -- [E007] Type Mismatch Error [error] |Found: (5 : Int) [error] |Required: eu.timepit.refined.api.Refined[Int, eu.timepit.refined.numeric.Positive]

It looks like the 2.x macros haven't been ported to Scala 3 yet.

[jmcclell]

I see this work is currently being experimented with in #921

[note]

I took a look at what it would take to migrate macros to Scala 3.

The biggest issue is stumbled upon is lack of equivalent of Evals.eval in Scala 3. eval is used here in refined. On this page that is considered a major blocker for migrating macros in a few projects. You can more for example here and here.

Anyway, I managed to create a very incomplete draft (https://github.com/fthomas/refined/pull/959), it supports only one use case and almost everything is hardcoded there. It shows the basic idea though - since we cannot evaluate the tree we have to manually pattern match against each possible predicate which is very cumbersome. Also, I don't see a way to support user defined custom predicates in this solution. Implementing non-leaf predicates (as e.g. And) might be tricky.

It's definitely a problematic solution, I would love to see something simpler.

The above attempt assumed we are constrained by current encoding. #921 suggests that @fthomas was experimenting with new encodings. I played around with Scala 3 features that seems relevant and you can see the outcome here. Predicates for Ints can be written exclusively in terms of scala.compiletime - link. With Strings writing macros are required but they are actually very straightforward - link. All in all, it looks promising. The biggest limitation of that encoding is that it uses closed world assumption - I don't see an easy way to support custom user-defined predicates.

A friend of mine hinted me about multi-stage programming. I've had no idea about it but it sounds like something worth looking into as a way of overcoming limitations of Scala 3 macros.

[mprevel]

Hi,

Is the issue still there ? I am trying to migrate a scala 2.13 project to scala 3 and refinements with refined.auto._ are not working anymore. Is there another way to do it ? It also seems that refineMV has been removed. If so it would be nice to add this limitation in the docs.

[jan0sch]

Still no solution I am aware of.

For test code you could circumvent it via val x = MyType.unsafeFrom(...) instead of val x: MyType = ... but I wouldn't recommend it for production code.

Some things can be done via opaque types but that re-involves a lot of boilerplate.

[armanbilge]

@kordyjan following up on the generous offer from your tweet, would greatly appreciate @virtuslab support in this effort! :)

[armanbilge]

So I experimented with the encoding from https://github.com/fthomas/refined/pull/921 in https://github.com/armanbilge/lucuma-core/pull/1 and it seems to work for the simple case there.

@note, can you expand on your comment in https://github.com/fthomas/refined/issues/932#issuecomment-833013736?

The biggest limitation of that encoding is that it uses closed world assumption - I don't see an easy way to support custom user-defined predicates.

Why is this the case? Since Predicate is an implicit, can't users just implement custom instances for whatever they want?

[note]

Hi @armanbilge,

The biggest limitation of that encoding is that it uses closed world assumption - I don't see an easy way to support custom user-defined predicates.

Why is this the case? Since Predicate is an implicit, can't users just implement custom instances for whatever they want?

In https://github.com/fthomas/refined/pull/959 I tried to stay the in realm of Validate and I didn't introduce Predicate there. It was an experiment and if it worked out out it would have an advantage that all other machinery relying on Validate would work out of the box.

I am writing of top of my head, probably @fthomas is the best person to ask about it, but from my analysis done months ago it looked like there's quite a few things, like type inference or treatment of Numeric relying on Validate. Basically, it looked to me Validate was a basic building block.

Now, what was tried out in https://github.com/fthomas/refined/pull/921 and in https://github.com/armanbilge/lucuma-core/pull/1 is putting another layer on top of existing stuff, namely Predicate. I agree it looks more promising but it would have to be developed further.

I wondered for a moment how Or would look like in your encoding:

// This is the client code we want to support:
val a = refineMV[Int, Or[Interval.Closed[3, 5], Interval.Closed[103, 105]]](4)

I was not able to write a type to make it compile in 3 minutes, if you can come up with this let me know. I will try to get back to this next days.

After sacrificing readability of types, I was able to figure it out this:

object Approach1 {
    inline given [T, A <: Predicate[T, _], B <: Predicate[T, _]]: Predicate[T, Or[A, B]] with
      transparent inline def isValid(inline t: T): Boolean = true // Dummy impl, just make it compile

    val a = refineMV[Int, Or[Predicate[Int, Interval.Closed[3, 5]], Predicate[Int, Interval.Closed[103, 105]]]](4)
  }

But, as you see, the type Predicate leaked to the user space, complicating the type. Also, I used a dummy implementation, it might be non-trivial to implement it, as the thing has to be fully inlined

What was nice from refined's POV in Scala 2 and, I believe it's not possible in Scala 3, is this: https://github.com/note/refined-demo/blob/main/refined2Base/src/main/scala/base/ContainsDollar.scala#L8. The ability to write Validate.Plain[String, YourType] only once and use it in both compile-time and runtime. It was possible only because of eval: https://github.com/fthomas/refined/blob/master/modules/core/shared/src/main/scala-3.0-/eu/timepit/refined/macros/MacroUtils.scala#L22

Anyway, it's good to revive this thread. And even if we cannot come up with how to write Or it may be still worth unblocking simple predicates at least?

[armanbilge]

@note is this what you are looking for?

//> using scala "3.1.2"
//> using lib "eu.timepit::refined::0.9.29"

import eu.timepit.refined.api.Refined
import eu.timepit.refined.boolean.Or
import eu.timepit.refined.numeric.Interval

import scala.compiletime.constValue

inline def refineMV[T, P](inline t: T)(using inline p: Predicate[T, P]): Refined[T, P] = {
  inline if (p.isValid(t)) Refined.unsafeApply(t) else scala.compiletime.error("no")
}

trait Predicate[T, P] {
  transparent inline def isValid(inline t: T): Boolean
}

object Predicate {

  inline given [M <: Int, N <: Int]: Predicate[Int, Interval.Closed[M, N]] with
    transparent inline def isValid(inline t: Int): Boolean = constValue[M] <= t && t <= constValue[N]

  inline given [T, A, B, PA <: Predicate[T, A], PB <: Predicate[T, B]](using predA: PA, predB: PB): Predicate[T, Or[A, B]] with
    transparent inline def isValid(inline t: T): Boolean = predA.isValid(t) || predB.isValid(t)
}

@main def main =
  println(refineMV[Int, Interval.Closed[1, 42]](42))
  println(refineMV[Int, Or[Interval.Closed[3, 5], Interval.Closed[103, 105]]](4))

[armanbilge]

Polite bump on this :) if folks are happy with this Predicate strategy I can work on a PR.

[armanbilge]

I see some 👍 so FYI I've begun assembling some predicates here on an as-needed basis: https://github.com/gemini-hlsw/lucuma-refined

The goal is that all of that can eventually be upstreamed.

[fthomas]

I should mention that I used the Predicate encoding in my experiment because it is much simpler than Validate which is current building block in refined. The idea was that if the refineMV macro can be implemented with Predicate, to try to the current Validate encoding next.

[armanbilge]

Aha, that's a very helpful pointer. Thanks!

At a glance it seems like Validate probably cannot be used as-is, since it lacks the necessary inline modifiers. But I guess the main idea is to mirror its more complete API :)

[longliveenduro]

As I am working with Scala 3.2 currently, it would be very nice if the caveats for Scala 3 would make it in the main Readme.

E,g, I stumbled upon "refineMV" missing.

[DmytroMitin]

@note

I took a look at what it would take to migrate macros to Scala 3. The biggest issue is stumbled upon is lack of equivalent of Evals.eval in Scala 3.

Actually there is kind of eval in Scala 3. There is eval from source code to value. Regarding eval from a tree to value it exists but deliberately blocked, to unblock the code expanding macros should be compiled with a patched compiler. https://github.com/DmytroMitin/dotty-patched

[erikerlandson]

Getting refineMV working in scala3 would also make the integration with coulomb nicer, but primarily I want it for unit testing, and I'm sure I can work around it. https://github.com/erikerlandson/coulomb/pull/392

[erikerlandson]

xref, I proposed pushing the literals into the types, since this is quite clean in scala3: https://github.com/fthomas/refined/issues/762

to wit: instead of refineMV[Positive](1) one could write refineMV[Positive, 1]

[erikerlandson]

+1 for @armanbilge inline / Predicate concept

[hejfelix]

Any news on this?