doobie icon indicating copy to clipboard operation
doobie copied to clipboard
verified publisher icon tpolecat

Code examples in docs/04-Selecting.html don't compile

Open mslinn opened this issue 6 years ago • 6 comments

I read that this was supposed to be impossible, but it is probably due to the code examples using a different version of the library. Using Ammonite (and its embedded Scala 2.12.8) I have this script:

import $ivy.`org.tpolecat::doobie-core:0.7.0`
import $ivy.`org.tpolecat::doobie-postgres:0.7.0`
import $ivy.`org.tpolecat::doobie-hikari:0.7.0`
import doobie._
import doobie.implicits._
import doobie.util.ExecutionContexts
import cats._
import cats.data._
import cats.effect.IO
import cats.implicits._
import fs2.Stream
implicit val cs = IO.contextShift(ExecutionContexts.synchronous)
val xa = Transactor.fromDriverManager[IO](
  "org.postgresql.Driver",     // driver classname
  "jdbc:postgresql:world",     // connect URL (driver-specific)
  "postgres",                  // user
  "",                          // password
  ExecutionContexts.synchronous // just for testing
)
case class Country(code: String, name: String, pop: Int, gnp: Option[Double])

So far so good. Now the problems start: I want to select country records from the database and save them into Country case class instances, then return the collection of case class instances.

sql"select code, name, population, gnp from country".
  query[Country].
  list.
  unsafeRunSync.
  transact(xa).
  unsafePerformIO

The error is:

cmd18.sc:3: value list is not a member of doobie.Query0[ammonite.$sess.cmd13.Country]
possible cause: maybe a semicolon is missing before `value list'?
  list.
  ^
Compilation Failed

I messed around typing in whatever incantations I could find, and got similarly unhelpful error messages. The Dooble API has changed a lot over the years, so most of the sample code on the interwebs no longer works. What magic incantation yields the complete collection of all records that match the select criteria?

mslinn avatar Jun 13 '19 06:06 mslinn

Query0#list is changed to Query0#to[List]. https://tpolecat.github.io/doobie/docs/04-Selecting.html#reading-rows-into-collections

Refer to the migration guide from 0.4 to 0.5: https://tpolecat.github.io/doobie/migration.html#miscellaneous-changes

guersam avatar Jun 13 '19 07:06 guersam

Yeah, I tried that too:

sql"select code, name, population, gnp from country".
    query[Country].
    to[List].
    transact(xa).
    unsafeRunSync
java.lang.NoSuchMethodError: cats.FlatMap.productR$(Lcats/FlatMap;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
  doobie.free.preparedstatement$$anon$2.productR(preparedstatement.scala:683)
  cats.Apply.$times$greater(Apply.scala:120)
  cats.Apply.$times$greater$(Apply.scala:11)
  doobie.free.preparedstatement$$anon$2.$times$greater(preparedstatement.scala:683)
  cats.Apply$Ops.$times$greater(Apply.scala:11)
  cats.Apply$Ops.$times$greater$(Apply.scala:11)
  cats.syntax.ApplySyntax$$anon$1.$times$greater(apply.scala:6)
  doobie.util.query$Query.to(query.scala:120)
  doobie.util.query$Query.to$(query.scala:119)
  doobie.util.query$Query$$anon$4.to(query.scala:211)
  doobie.util.query$Query$$anon$3.to(query.scala:192)
  ammonite.$sess.cmd18$.<init>(cmd18.sc:3)
  ammonite.$sess.cmd18$.<clinit>(cmd18.sc)

mslinn avatar Jun 13 '19 07:06 mslinn

I don't understand what I'm doing, but this seems to be getting closer to the desired result:

sql"select code, name, population, gnp from country".
      query[Country].stream.transact(xa).compile.to[List]
res35: IO[List[Country]] = Map(
  Bind(Delay(fs2.Stream$CompileOps$$Lambda$3374/0x0000000840e59840@1a65732e), fs2.Stream$Compiler$$anon$8$$Lambda$3379/0x0000000840e5c840@fb24b47),
  fs2.Stream$CompileOps$$Lambda$3376/0x0000000840e5b040@60de9df8,
  0
)

Adding unsafeRunSync, whatever that does, causes an error:

@  sql"select code, name, population, gnp from country".
      query[Country].stream.transact(xa).compile.to[List].unsafeRunSync
java.lang.NoSuchMethodError: cats.FlatMap.productR$(Lcats/FlatMap;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
  doobie.free.preparedstatement$$anon$2.productR(preparedstatement.scala:683)
  cats.Apply.$times$greater(Apply.scala:120)
  cats.Apply.$times$greater$(Apply.scala:11)
  doobie.free.preparedstatement$$anon$2.$times$greater(preparedstatement.scala:683)
  cats.Apply$Ops.$times$greater(Apply.scala:11)
  cats.Apply$Ops.$times$greater$(Apply.scala:11)
  cats.syntax.ApplySyntax$$anon$1.$times$greater(apply.scala:6)
  doobie.hi.connection$.prepared$1(connection.scala:48)
  doobie.hi.connection$.$anonfun$liftStream$3(connection.scala:55)
  doobie.hi.connection$.$anonfun$liftStream$3$adapted(connection.scala:55)
  fs2.Stream$.$anonfun$flatMap$1(Stream.scala:1038)
  fs2.internal.FreeC.$anonfun$flatMap$1(FreeC.scala:33)
  fs2.internal.FreeC$ViewL$.mk(FreeC.scala:214)
  fs2.internal.FreeC$ViewL$.apply(FreeC.scala:206)
  fs2.internal.FreeC.viewL(FreeC.scala:67)
  fs2.internal.Algebra$.translateStep$1(Algebra.scala:410)
  fs2.internal.Algebra$.$anonfun$translate0$2(Algebra.scala:446)
  fs2.internal.FreeC.$anonfun$transformWith$1(FreeC.scala:43)
  fs2.internal.Algebra$.$anonfun$compileLoop$10(Algebra.scala:260)
  fs2.internal.Algebra$.$anonfun$compileLoop$1(Algebra.scala:230)
  cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:139)
  cats.effect.internals.IORunLoop$RestartCallback.signal(IORunLoop.scala:351)
  cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:372)
  cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:312)
  cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:136)
  cats.effect.internals.IORunLoop$RestartCallback.signal(IORunLoop.scala:351)
  cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:372)
  cats.effect.internals.IORunLoop$RestartCallback.apply(IORunLoop.scala:312)
  cats.effect.internals.IOShift$Tick.run(IOShift.scala:36)
  doobie.util.ExecutionContexts$synchronous$.execute(ExecutionContexts.scala:33)
  cats.effect.internals.IOShift$$anon$1.apply(IOShift.scala:28)
  cats.effect.internals.IOShift$$anon$1.apply(IOShift.scala:26)
  cats.effect.internals.IORunLoop$RestartCallback.start(IORunLoop.scala:337)
  cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:119)
  cats.effect.internals.IORunLoop$.startCancelable(IORunLoop.scala:41)
  cats.effect.internals.IOBracket$BracketStart.run(IOBracket.scala:86)
  cats.effect.internals.Trampoline.cats$effect$internals$Trampoline$$immediateLoop(Trampoline.scala:70)
  cats.effect.internals.Trampoline.startLoop(Trampoline.scala:36)
  cats.effect.internals.TrampolineEC$JVMTrampoline.super$startLoop(TrampolineEC.scala:93)
  cats.effect.internals.TrampolineEC$JVMTrampoline.$anonfun$startLoop$1(TrampolineEC.scala:93)
  scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
  scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
  cats.effect.internals.TrampolineEC$JVMTrampoline.startLoop(TrampolineEC.scala:93)
  cats.effect.internals.Trampoline.execute(Trampoline.scala:43)
  cats.effect.internals.TrampolineEC.execute(TrampolineEC.scala:44)
  cats.effect.internals.IOBracket$BracketStart.apply(IOBracket.scala:72)
  cats.effect.internals.IOBracket$BracketStart.apply(IOBracket.scala:52)
  cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:136)
  cats.effect.internals.IORunLoop$.start(IORunLoop.scala:34)
  cats.effect.internals.IOBracket$.$anonfun$apply$1(IOBracket.scala:44)
  cats.effect.internals.IOBracket$.$anonfun$apply$1$adapted(IOBracket.scala:34)
  cats.effect.internals.IORunLoop$RestartCallback.start(IORunLoop.scala:337)
  cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:119)
  cats.effect.internals.IORunLoop$.$anonfun$suspendInAsync$1(IORunLoop.scala:247)
  cats.effect.internals.IORunLoop$.$anonfun$suspendInAsync$1$adapted(IORunLoop.scala:246)
  cats.effect.internals.IORunLoop$RestartCallback.start(IORunLoop.scala:337)
  cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:119)
  cats.effect.internals.IORunLoop$.start(IORunLoop.scala:34)
  cats.effect.IO.unsafeRunAsync(IO.scala:258)
  cats.effect.internals.IOPlatform$.unsafeResync(IOPlatform.scala:38)
  cats.effect.IO.unsafeRunTimed(IO.scala:325)
  cats.effect.IO.unsafeRunSync(IO.scala:240)
  ammonite.$sess.cmd34$.<init>(cmd34.sc:2)
  ammonite.$sess.cmd34$.<clinit>(cmd34.sc)

mslinn avatar Jun 13 '19 07:06 mslinn

It looks like you have some library conflicts that may be specific to Ammonite weirdness. Does it work in a normal Scala project?

tpolecat avatar Jun 13 '19 11:06 tpolecat

I had some success. I documented the issues in the DoobieDoo github project. Update: I fixed the build script and gave up on Ammonite.

The major issue remaining is that the types for the following two transactors (xaSynch and xaAsynch) are dissimilar and are not interchangeable:

  /** Single threaded, just for testing */
  val xaSynch: Aux[IO, Unit] = Transactor.fromDriverManager[IO](
    "org.postgresql.Driver",      // driver classname
    "jdbc:postgresql:world",      // connect URL (driver-specific)
    "postgres",                   // username
    "",                           // password
    ExecutionContexts.synchronous
  )

  /** Multi-threaded transactional resource. Modified for Postgres from
    * https://tpolecat.github.io/doobie/docs/14-Managing-Connections.html#using-a-hikaricp-connection-pool */
  val xaAsynch: Resource[IO, HikariTransactor[IO]] =
    for {
      connectEC     <- ExecutionContexts.fixedThreadPool[IO](32)
      transactionEC <- ExecutionContexts.cachedThreadPool[IO]
      xa            <- HikariTransactor.newHikariTransactor[IO](
        "org.postgresql.Driver",   // driver classname
        "jdbc:postgresql:world",   // connect URL (driver-specific)
        "postgres",                // username
        "",                        // password
        connectEC,                 // await connection here
        transactionEC              // execute JDBC operations here
      )
    } yield xa

This works:

  val result: Seq[Country] = sql"select code, name, population, gnp from country"
    .query[Country]
    .to[List]
    .transact(xaSynch)
    .unsafeRunSync

... but replacing xaSynch with xaAsynch does not compile:

  val result: Seq[Country] = sql"select code, name, population, gnp from country"
    .query[Country]
    .to[List]
    .transact(xaAsynch)
    .unsafeRunSync

Error:(76, 15) type mismatch;
 found   : cats.effect.Resource[cats.effect.IO,doobie.hikari.HikariTransactor[cats.effect.IO]]
    (which expands to)  cats.effect.Resource[cats.effect.IO,doobie.util.transactor.Transactor[cats.effect.IO]{type A = com.zaxxer.hikari.HikariDataSource}]
 required: doobie.util.transactor.Transactor[?]
    .transact(xaAsynch)

The documentation focuses on using xaSynch and does not show a code example with any other transactor. That's probably how this problem was not noticed before.

On a related note, why doesn't insert1 also invoke .run.transact(xaSynch).unsafeRunSync instead of returning an instance of Update0? Is this for optimizing by batching transactions together? If so, then some mention of that would be helpful in the docs, along with a code example.

mslinn avatar Jun 13 '19 16:06 mslinn

I figured out a way forward. Seems that xaAsynch is not a dropin replacement for xaSynch. This code works, however:

  val result: Seq[Country] = xaAsynch.use { xa =>
    sql"select code, name, population, gnp from country"
    .query[Country]
    .to[List]
    .transact(xa)
  }.unsafeRunSync()

The other change I had to make was to rename the pop property of Country case class to match the SQL column name population.

All of this should be added to the docs.

Quill does a nice job of mapping SQL columns to case class properties. I just found the docs for the Doobie/Quill integration, will check it out.

mslinn avatar Jun 13 '19 20:06 mslinn