config-rs icon indicating copy to clipboard operation
config-rs copied to clipboard

failed type conversion in .ini file for serde flatten

Open mfirhas opened this issue 1 year ago • 10 comments

I think other formats are all fine, but for .ini/.env, when I flatten the structs into another struct, type conversion failed. I checked the code and found this:

https://github.com/mehcode/config-rs/blob/b3bda2c37ea4823b6842ce0f08fd281c07374d60/src/file/format/ini.rs#L21 https://github.com/mehcode/config-rs/blob/b3bda2c37ea4823b6842ce0f08fd281c07374d60/src/file/format/ini.rs#L30

All code are string-ed rightaway.

Should it be parsed like this?: https://github.com/mehcode/config-rs/blob/b3bda2c37ea4823b6842ce0f08fd281c07374d60/src/env.rs#L282

I test locally and it works and was about to create PR, but confuse with master branch status I faced unexpected error, unlike in v0.13.4.

mfirhas avatar Dec 13 '23 00:12 mfirhas

Hi, thanks for reporting this! A PR would be awesome of course!

, but confuse with master branch status I faced unexpected error,

What do you mean by this?

matthiasbeyer avatar Dec 13 '23 06:12 matthiasbeyer

I created my own test code, test it on master branch got error missing field 'field_name', switched to v0.13.4 got no error. The test was for loading configs from .env/.ini file.

mfirhas avatar Dec 13 '23 12:12 mfirhas

The last CI run on master is green though.

matthiasbeyer avatar Dec 13 '23 12:12 matthiasbeyer

Here is my test code:

// tests/file_env_test.rs

use config::{
    builder::DefaultState, Config, ConfigBuilder, ConfigError, Environment, File, FileFormat,
};
use serde::Deserialize;
use std::fmt::{Debug, Display};

const ALL_CFG_FILE: &str = "./tests/all.env";

#[derive(Debug, Deserialize)]
struct TestConfig {
    #[serde(alias = "TEST__NUMBER")]
    pub number: i32,
    #[serde(alias = "TEST__STRING")]
    pub string: String,
    #[serde(alias = "TEST__FLOAT")]
    pub float: f32,
    #[serde(alias = "TEST__BOOLEAN")]
    pub boolean: bool,
}

#[test]
fn test_file_config_success() {
    let cfg = Config::builder()
        .add_source(config::File::new(ALL_CFG_FILE, FileFormat::Ini))
        .build()
        .unwrap()
        .try_deserialize::<TestConfig>();
    dbg!(&cfg);
    assert!(cfg.is_ok());
}

ini file tests/all.env

TEST__NUMBER=123
TEST__STRING=this is string from file
TEST__FLOAT=123.3
TEST__BOOLEAN=true

My original issue is about type incompatibility on nested configs parsing from .ini file. Above test code is only to test between master and v0.13.4, but found said error(missing field) on master.

mfirhas avatar Dec 13 '23 13:12 mfirhas

There're many commits unmerged from v0.13.4 into master, hence different version.

mfirhas avatar Dec 15 '23 01:12 mfirhas

This is the my commit id: e30debf7cab0b4b8e0edbd254cdc58edc24ab7b2

mfirhas avatar Dec 15 '23 01:12 mfirhas

All code are string-ed rightaway.

For reference the INI support is being refactored a little here: https://github.com/mehcode/config-rs/pull/470

That doesn't change the logic you draw attention to though. The reason we are calling .to_owned() there is because rust-ini is used and it provides an iterator over &str, there is no type information available. To "import" that data we need to create a copy of the strings it provides.

The other formats often parse the strings themselves and provide a Value enum that can then be mapped into the equivalent for config-rs (which normalizes Value type for internal use).

Should it be parsed like this?:

Yes, that sort of improvement could probably be added to (or after) the referenced refactor PR. It is intended for 0.14 so as a breaking change would be ok AFAIK, @matthiasbeyer ?

It can be a bit problematic depending on the value I think 🤷‍♂️

  • A "0" or "1" could "deserialize" into a bool that can be converted to 0/1 number type of a config struct.
  • But a string that intentionally begins with 0 like a phone number would lose that information, even if the config struct from the user is a string type? I think this is why you'll find other formats that do provide a value type tend to map into an equivalent config-rs value, so rust-ini providing strings and converting to strings still seems appropriate.

Here a string to a float:

https://github.com/mehcode/config-rs/blob/b3bda2c37ea4823b6842ce0f08fd281c07374d60/src/value.rs#L521-L540

and likewise string (or whatever other intermediary value in our container) to a bool type of the user config struct:

https://github.com/mehcode/config-rs/blob/b3bda2c37ea4823b6842ce0f08fd281c07374d60/src/value.rs#L233-L245

I think the issue is more related to that area, than rust-ini and if you attempt to fix it with the ini format you risk introducing regressions and less flexibility 😅

polarathene avatar Dec 15 '23 01:12 polarathene

Okay, then I'll just need to use that version after release.

mfirhas avatar Dec 15 '23 01:12 mfirhas

when I flatten the structs into another struct, type conversion failed

I believe there are other issues related to serde deserializing (the internal value.rs representation, into user struct) within the project?

  • https://github.com/mehcode/config-rs/issues/88
  • https://github.com/mehcode/config-rs/issues/336
  • Potentially related issue for #[serde(rename=value)] due to internal feature: https://github.com/mehcode/config-rs/issues/417#issuecomment-1770379776
  • Another issue with deserializing (proposes solution, but changes since may have affected the viability): https://github.com/mehcode/config-rs/issues/461#issuecomment-1771963758

I'll just need to use that version after release.

I don't know if your concern will be fixed. Addressing INI to parse from &str to a ValueKind may fix your concern, but replace it with a regression that impacts others, which is not really a fix.

The value should already be captured into Value (value.rs) as a String and from there to the desired type. The fix you want should be handled there instead, but as shown above there are some cases where serde or the internal usage of it within config-rs is not compatible with some serde attributes like #[serde(flatten)].

Some formats are also known to be lossy in conversion into the config-rs Value container.

polarathene avatar Dec 15 '23 01:12 polarathene

For now, I'll just refer to my commit in my fork. After this is fixed/refactored as a whole, will switch back.

mfirhas avatar Dec 15 '23 02:12 mfirhas