Config files without EOF newline in dependencies causes parsing errors

[Naillik1]

Hi! Came across this problem with config file concatenation, and it seems like a bug -- if it's the intended behaviour, I'd love to know where I went wrong.

---

Problem

I have two Scala projects: app & lib, where lib is included as a dependency of app. Both app & lib depend on typesafe config 1.4.0 via PureConfig 0.13.0. Each has a reference.conf.

Recently, an end-of-file newline was removed from lib/../reference.conf, causing various parsing errors upon attempting to load configs in app. Which particular error is thrown depends on the contents of the first line of app/.../reference.conf.

---

Examples

Base

// app/.../Main.scala
import com.typesafe.config.ConfigFactory
object Main extends App {
    ConfigFactory.load()
}

The following examples are the results of varying either lib/.../reference.conf or app/.../reference.conf, then re-building and running app (sbt assembly; java -jar /out/app/assembly/dest/out.jar)

1. Primitive on first line of app/.../reference.conf yields ConfigException: Parse

# lib/.../reference.conf 
setting = snag # <EOF, no newline>

# app/.../reference.conf
foo = bar

com.typesafe.config.ConfigException$Parse:
reference.conf @ jar:file:/out/app/assembly/dest/out.jar!/reference.conf: 2713: 
Expecting end of input or a comma, got '=' (if you intended '=' to be part of a key or string value, try enclosing the key or value in double quotes, or you may be able to rename the file .properties rather than .conf)

• Debugging this was entirely opaque due to the error message referring to the line-number in the concatenated file set; essentially a matter of trial-and-error removing configs until the issue could be isolated to the first line of the file.

2. Object on first line of app/.../reference.conf yields ConfigException: WrongType

# lib/.../reference.conf 
setting = snag # <EOF, no newline>

# app/.../reference.conf
foo {
  bar = "baz"
}

com.typesafe.config.ConfigException$WrongType: 
reference.conf @ jar:file:/out/app/assembly/dest/out.jar!/reference.conf: 2713:
Cannot concatenate object or list with a non-object-or-list, Unquoted("snagfoo") and SimpleConfigObject({"bar":"baz"}) are not compatible

• I was lucky enough to both own the offending lib code and to recognize that snag was a config value found in lib. Without both, I'd be no closer to knowing what's going on.

3. Inserting a newline at first line of app/.../reference.conf properly loads config

# lib/.../reference.conf 
setting = snag # <EOF, no newline>

# app/.../reference.conf

foo = bar

4. Inserting a newline at lib/.../reference.conf end-of-file properly loads config

# lib/.../reference.conf
setting = snag

# app/.../reference.conf
foo = bar

---

I'd expect config file concatenation to insert a break between files to guard against this; is there a reason why concatenating config files doesn't do this? Or is there some other mechanism I'm missing that I should be doing?

If it's indeed a bug, I'm happy to contribute a fix (though I'd appreciate any pointers as to where to start).

[havocp]

I don't think the config library itself does textual concatenation- it may be the problem here is in a fat jar build plugin that concats files textually? maybe it could add a newline?

[raboof]

So what you're seeing is the reference.conf files get concatenated without inserting newlines.

This is not something typesafe-config can help: the files are concatenated by your build tool - in this case, by the sbt-assembly plugin.

I remember seeing the behavior you describe (and making sure my reference.conf files have a newline at the end). However, looking at the sources for the 'concat' merge strategy of sbt-assembly, it seems it should be inserting a newline if there is none: https://github.com/sbt/sbt-assembly/blob/v0.15.0/src/main/scala/sbtassembly/MergeStrategy.scala#L66 . That code seems to have been there for quite a while, too.

If you expect your library to be used in projects that are not using sbt, it might be smart to just make sure the newline is there - even if you fix sbt-assembly, other build tools/plugins might be harder to configure to insert those newlines.

If it's indeed a bug, I'm happy to contribute a fix (though I'd appreciate any pointers as to where to start).

This looks like it might be a bug in sbt-assembly, so I'd start looking closer there.

[Naillik1]

@raboof Ah, that looks to be it -- I've been using mill, and taking a closer look concatenation during mill's assembly task appears to be the issue. Thanks for pointing me in the right direction!