Why it's not possible to create a Config from a JSON Array?

[asko]

Hi,

Originally I've asked for help at stackoverflow but few minutes back realized that it would be a lot better to ask here.

I have a problem of not being able to parse array-structured JSON into the Config. According to HOCON definition arrays at the root level are totally valid, and ConfigDocumentFactory.parseReader(..) is actually parsing an array without issues. However, when I try to parse an array into the Config using ConfigFactory.parseReader(..) I get this:

com.typesafe.config.ConfigException$WrongType: Reader: 1:  has type LIST rather than object at file root
	at com.typesafe.config.impl.Parseable.forceParsedToObject(Parseable.java:136)
	at com.typesafe.config.impl.Parseable.parse(Parseable.java:299)
	at com.typesafe.config.ConfigFactory.parseReader(ConfigFactory.java:622)
	at com.typesafe.config.ConfigFactory.parseReader(ConfigFactory.java:636)

Is it some limitation of the Config? Is there a way to work it around (or get a Config instance with a fake root out of parsed ConfigDocument)?

I wish I could change JSON array to an object (which would get parsed without issues, but unfortunately I get this array from the response of a web service beyond my control...

[havocp]

A Config is an object, so while you can parse json/hocon arrays, they can't be returned via APIs that return a Config. You would need an API that returns ConfigValue.

As far as I remember offhand the library may not have public API for that since it wouldn't afaik be useful for configuration purposes. The library isn't intended for use as a generic data interchange/JSON parser - it's probably the slowest JSON parser on earth, aside from API limitations.

Anyway short answer non-object roots are valid json/hocon but not valid config files, according to typesafe config.

[havocp]

If you wanted to add a parseReader method to ConfigValueFactory that seems ok. Or something along those lines? The needed code is there, you can see in the stack trace that it already parses the array but then throws an error because it has to return a Config.

[asko]

Thanks @havocp for advice to look at ConfigValue! I got my problem solved with slight use of reflection (to access package-private Parseable.parseValue()):

    @Test
    public void config_howCouldItParseArray() throws Exception {
      String jsonArray = "[{'element':'value'}]".replace("'","\"");
      Parseable p = Parseable.newReader(new StringReader(jsonArray), ConfigParseOptions.defaults().setSyntax(ConfigSyntax.JSON));
      ConfigValue v = reflectionParseValue(p);
      ConfigList configList = (ConfigList) v;
      Assert.assertEquals(configList.size(),1);
    }

    private ConfigList reflectionParseValue(Parseable p) throws Exception {
      Method parseValue = Parseable.class.getDeclaredMethod("parseValue");
      parseValue.setAccessible(true);
      ConfigValue configValue = (ConfigValue) parseValue.invoke(p);
      if (configValue.valueType() != ConfigValueType.LIST) {
        throw new IllegalArgumentException("Parsed value is not of a LIST type");
      }
      
      return (ConfigList) configValue;
    }

Re making it a part of an API - three questions:

1. would it be enough to add two ConfigValue ConfigValueFactory.parseReader(Reader reader) and ConfigValue ConfigValueFactory.parseReader(Reader reader, ConfigParseOptions options) methods?

2. where to put tests to (I've checked ConfigValueTest.scala and PublicApiTest.scala, latter one seems to fit better)?

3. would it be ok to make .parseValue() of Parseable public?

[havocp]

Yes, I think adding just the reader parser to ConfigValueFactory might be fine; people can always create their own reader from a file or whatever, so we don't need all the convenience methods like parseFile.

PublicApiTest seems like a good place. Be sure to test parsing arrays :-)

I would maybe add parseValue to the ConfigParseable interface (and then of course make it public).

[havocp]

on second thought I'm not sure parseValue needs to be in ConfigParseable but I guess it does need to be public. I think ConfigParseable is only used to include objects.

[asko]

yeah, I have the same concern. Another option would be extracting ConfigValueParseable related functionalty out of Parseable implementations, but I have no idea is it worth long-term. So, just to make sure, I'm adding ConfigValue parseValue(Reader ...) to ConfigParseable?

[havocp]

Using Parseable.parseValue directly from ConfigValueFactory hopefully works. I think ConfigParseable is only public for use in ConfigIncluder implementations or something obscure like that.

[pjrt]

Has there been any work on this? Would be great to have this.

[asko]

Drowning in direct responsibilities, hope to have this through in September. Thanks for reminding.

чт, 31 авг. 2017 г. в 21:27, Pedro Rodriguez Tavarez < [email protected]>:

Has there been any work on this? Would be great to have this.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/typesafehub/config/issues/460#issuecomment-326382208, or mute the thread https://github.com/notifications/unsubscribe-auth/ARJqpx8DPCoTkGra2kmbywCv9oGJmlVeks5sdvr1gaJpZM4MYyjF .

-- asko

[pushpendra-jaiswal-90]

Is it resolved ?

[dwijnand]

Anyway short answer non-object roots are valid json/hocon but not valid config files, according to typesafe config.

While I've known there was always a difference between the generic HOCON parts and the config specific parts (such as reference.conf/application.conf), this particular distinction between what is HOCON data and what is more specifically config data wasn't clear to me until now.

Perhaps "ConfigValue" should've been HoconValue or something along those lines.

[dwijnand]

Oh and the "Config" part of "Human-Optimized Config Object Notation" also makes this distinction less obvious.

[kell18]

Hi! Any news?