zio-config icon indicating copy to clipboard operation
zio-config copied to clipboard

Add ConfigSource#toLayer

Open jdegoes opened this issue 3 years ago • 2 comments

Currently, wrapping a ConfigSource into a layer will do the wrong thing.

e.g. fromHocon(...).toLayer.

The problem is that the memoization happens in the "outer" ZManaged of the config source. But layers require memoization happen as part of layer construction.

In order to address this, we can add a tolayer method to ConfigSource, which shifts the outer layer into the ZLayer, so that it will behave properly, according to user expectations.

trait ConfigSource {
  def toLayer: ZLayer[Any, Nothing, Has[ConfigSource]] = ???
  ...
}

jdegoes avatar Nov 02 '21 18:11 jdegoes

@jdegoes Could you elaborate a bit more on this? Example, Given we have the following implementation of properties-file ConfigSource (where resource acquisition and release is in terms of ZManaged) what's the implication of using ZLayer?


val x : ZIO[Any, ReadError[K], MyConfig] = read(config from fromPropertiesFile(path))

def propertiesFile(
  filePath: String,
  keyDelimiter: Option[Char] = None,
  valueDelimiter: Option[Char] = None,
  filterKeys: String => Boolean = _ => true
): ConfigSource = {
      val managed: ZManaged[Any, ReadError[K], PropertyTreePath[String] => UIO[PropertyTree[String, String]]] =
        ZManaged
          .make({
            ZIO.effect(
              new FileInputStream(new File(filePath))
            ).flatMap(properties => ZIO.effect {
              val properties = new java.util.Properties()
               properties.load(inputStream)
               properties
            })
          }) { r => ZIO.effectTotal(r.close()) }
          .map { tree =>  (path: PropertyTreePath[K]) => ZIO.succeed(tree.at(path)) }
          .mapError(throwable => ReadError.SourceError(throwable.toString))

      Reader(
        Set(ConfigSourceName(filePath)),
        ZManaged.succeed(managed)
      )
    }


User can also do read(config from fromPropertiesFile(path).memoized) which implies if the same config-source is referred multiple times during the read, the resource management is memoized (or the effect that is required to retrieve the ConfigSource is memoized)

afsalthaj avatar Nov 06 '21 09:11 afsalthaj

@jdegoes As of now what I have as toLayer is this:

https://github.com/zio/zio-config/blob/e903eea859f5c8621db52b80f859b63f25ecf504/core/shared/src/main/scala/zio/config/ConfigSourceModule.scala#L71

    /**
     * With `strictlyOnce`, regardless of the number of times `read`
     * is invoked, `ConfigSource` is evaluated
     * strictly once.
     *
     * It returns an Effect, because by the time ConfigSource is retrieved,
     * an effect is performed (which may involve a resource acquisition and release)
     *
     * {{{
     *   val sourceZIO = ConfigSource.fromPropertiesFile(...).strictlyOnce
     *
     *   for {
     *     src     <- sourceZIO
     *     result1 <- read(config from src)
     *     result2 <- read(config from src)
     *   } yield (result1, result2)
     *
     * }}}
     *
     * In this case, the propertiesFile is read only once.
     *
     * vs
     *
     * {{{
     *   val source: ConfigSource =
     *     ConfigSource.fromPropertiesFile(...).memoize
     *
     *   for {
     *     result1 <- read(config from source)
     *     result2 <- read(config from source)
     *   } yield (result1, result2)
     *
     * }}}
     *
     * In this case, the propertiesFile is read once per each read, i.e, twice.
     */
    def strictlyOnce: ZIO[Any, ReadError[K], ConfigSource] =
      (self match {
        case ConfigSource.OrElse(self, that) =>
          self.strictlyOnce.orElse(that.strictlyOnce)

        case ConfigSource.Reader(names, access) =>
          val strictAccess = access.flatMap(identity).use(value => ZIO.succeed(value))
          strictAccess.map(reader => Reader(names, ZManaged.succeed(ZManaged.succeed(reader))))
      })

    /**
     * A Layer is assumed to be "memoized" by default, i.e the construction
     * of ConfigSource layer is done strictly once regardless of number times the read is invoked.
     */
    def toLayer: ZLayer[Any, ReadError[K], Has[ConfigSource]] =
      strictlyOnce.toLayer

afsalthaj avatar Nov 07 '21 05:11 afsalthaj