Parse error

[ewinz87]

[error] toml::parse_table: invalid line format
--> C:\test.toml
179 | x = 2.68

This is end of file with no "LF" or "CRLF" at the end of last line. With "CRLF" at the end of last line I get this:

[error] toml::parse_key: the next token is not a key

With "LF" line ending it parses fine. Problem is that my Windows users and their editors will probably use "CRLF"...

test.toml contains a few tables and "CRLF" line ending.

I'm using Visual Studio 2017. My project is on C++17. Any help would be appreciated, everything else seems to work fine.

I guess the problem seems to be with unterminated strings and "CRLF".

One thing that did get my attention is that getting a float that's say "1" instead of "1.0" throws an error. It would be great if it parsed the "1" and cast it to "1.0" so the config file is even more noob friendly. I'm a beginner so I don't know if it's a limitation. Anyways, thanks for your work and sharing!

[ToruNiina]

Thank you for telling me the issue!

But sorry I couldn't reproduce a problem you encountered. I made a file in the following way.

#!/bin/bash
printf "key = \"value\"\r\n" >  sample.toml
printf "[table]\r\n"         >> sample.toml
printf "y = 42\r\n"          >> sample.toml
printf "x = 2.68"            >> sample.toml

And I checked that it does not contain any LF/CRLF at the end of the file by xxd. Unfortunately, I don't have any Windows machine, so I tried to reproduce your error by using gcc 8.2 with -std=c++17. And my sample code works.

From your description, the problem should be around this portion of the code. But from here, I couldn't get the reason. Could you tell me some more information like which version (or commit) did you used and/or what kind of error message followed after the line (e.g. "expected newline, but got '0xFF'")?

---

I'm sorry for the inconvenience about the conversion from Integer value to Float. But it is not supported by toml::get.

The reason why getting value "1" as double fails is a bit complicated. Firstly, In TOML, "1" is considered as Integer, not Float. Since toml::float is an alias of double, toml::get<double>(toml::value&) returns reference to the value contained. If I enable toml::get to convert Integer value to Float, the function cannot return a (lvalue) reference because toml::get create a value inside depending on the run-time condition. It disables a code like the following.

toml::get<toml::Float>(v) = 3.14;

I also sometimes think it is nice to make it convertible, but it is technically difficult to achieve both convertibility and mutability in the same function. Because I have a plan to implement serialization of toml::value (as Boost.toml has), I don't want to give up mutability.

It may be good to implement one more function that does not return a reference but automatically converts the type of value as a user wants...

[ToruNiina]

I found some bugs in parse_ml_table. Although it is not completely the same as your case, it is really similar to your case. I hope it fixes the problem you encountered.

The bugs I found occurs when a file ends with a line that contains extra whitespaces or comment but no LF/CRLF. I fixed the bugs and added the minimal case to test codes. Now I'm waiting for all the automatic tests performed in #17 pass.

Could you try the latest version after I merge #17 ?

[ewinz87]

Happy holidays!

I tested again with the newest version and the problem isn't resolved. I spent some time to narrow the issue and came up with the following conclusions:

It appears the position of "CRLF" in the file is irrelevant. I had a file with just 1 "CRLF" and rest just line feeds and it couldn't parse.

While debugging I noticed, if there's a "CRLF" anywhere, when it reaches the end, the const_iterator '_iter' in struct 'location' return "" so it tries to parse nothing really and this is when the error occurs.

I don't know if this helps. This is above my skill level. For now I'm using a quick and dirty work around, I replace all "CRLF"s by "LF" before parsing.

Will report if I find more information!

[ToruNiina]

Hi,

Sorry for taking long, but I finally succeed to reproduce the problem you reported. I got the same error on AppVayor while reading toml files (you can see it in #24 ). And I fixed it in #25 .

The reason is the following. C-streams that are opened as text-mode automatically converts CRLF to LF while reading (actually I didn't know that until today), and it causes inconsistency between the file size and the number of characters that would actually be read. As a result, the buffer contains uninitialized characters around the last element and it causes the error.

I changed the default openmode in toml::parse and the error disappeared (at least on AppVayor). @ewinz87 , could you try it if you have time?

[plevold]

Old issue, but I'm pretty sure this works as intended now. I encountered this issue when passing an std::ifstream to toml::parse instead of letting toml11 open the file itself.

Specifically my code effectively did

auto is = std::ifstream(filename);
auto data = toml::parse(is, filename);

which gave me this error. I resolved it by instead doing

auto is = std::ifstream(filename, std::ios_base::binary);
auto data = toml::parse(is, filename);

The following also works:

auto data = toml::parse(filename);

Of course this is well documented in the README, I just missed the crucial part... 😁

[ToruNiina]

Thank you very much @plevold , now I can close this.