envy
envy copied to clipboard
Serde flatten
🐛 Bug description
Using the flatten attribute from serde almost works but breaks in the case of non string values in flattened structs. In this case config always parses size as a string. However, if I put the size attribute directly in Config then everything works.
🤔 Expected Behavior
The usize value in the flattened struct should parse
👟 Steps to reproduce
#[derive(Deserialize)]
struct Config {
#[serde(flatten)]
pub subconfig: Subconfig
}
#[derive(Deserialize)]
struct Subconfig {
pub size: usize
}
🌍 Your environment
nightly-x86_64-unknown-linux-gnu (default) rustc 1.33.0-nightly (a8a2a887d 2018-12-16)
envy version: latest
I have also noticed this bug with bool variables.
@glademiller @xoac running into the same issue, did you find a workaround?
Also looking for a solution to this.
I'm open to pull requests to add this feature
I tried giving it a look but couldn't even locate the bug, any tips?
I spent some time trying to trace down where the issue is with this. Turns out #[serde(flatten)]
deserializes the sub-structure as a map, which means it only works well for self-describing formats (e.g. JSON), since it will always defer to deserialize_any
.
I haven't been able to come up with a good workaround yet.
Faced this isssue today, went with manual deserializing:
use serde::de;
use std::{fmt, fmt::Display, marker::PhantomData, str::FromStr};
pub fn deserialize_stringified_any<'de, D, T>(deserializer: D) -> Result<T, D::Error>
where
D: de::Deserializer<'de>,
T: FromStr,
T::Err: Display,
{
deserializer.deserialize_any(StringifiedAnyVisitor(PhantomData))
}
pub struct StringifiedAnyVisitor<T>(PhantomData<T>);
impl<'de, T> de::Visitor<'de> for StringifiedAnyVisitor<T>
where
T: FromStr,
T::Err: Display,
{
type Value = T;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("a string containing json data")
}
fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
where
E: de::Error,
{
Self::Value::from_str(v).map_err(E::custom)
}
}
And then:
#[derive(Deserialize)]
struct Config {
#[serde(flatten)]
pub fluentd_config: FluentdConfig
}
#[derive(Deserialize, Debug)]
pub struct FluentdConfig {
pub fluentd_host: String,
#[serde(deserialize_with = "deserialize_stringified_any")]
pub fluentd_port: u16,
pub fluentd_environment: String,
pub fluentd_tag: String,
}
Not ideal but works to me
I worked around by creating non-flattened struct and mapping that to desired after parsing environment variables, seemed easier than writing custom deserializers.
Any new advice besides implementing deserialize_stringified_any
and it's visitor? Thanks @Pzixel for the quick fix, did you just know that would work because you know serde well or is that some kind of escape hatch?
I decided to go with not making my root configuration struct deserializable, and then using a prefix for each nested struct inside something like this:
#[derive(Deserialize, Serialize, Debug)]
pub struct DBConfig {
host: String,
port: u16,
user: String,
pass: String,
}
#[derive(Debug)]
pub struct Env {
pub db: DBConfig,
}
impl Env {
pub fn from_env() -> Result<Self, Report> {
let db = envy::prefixed("PG").from_env::<DBConfig>()?;
Ok(Self { db })
}
}
(eg.) I pass in values as standard PostgreSQL envs and get back the correct values!: PGPASS=***
2022-03-19T09:56:16.719152Z INFO example_envy: env=Env { db: DBConfig { host: "localhost", port: 5432, user: "psql", pass: "psql" } }
I think @nazar-pc might have been hinting at something similar?
Any update on that one, it seems like if the nested type is always expected to be a string,
Tried to put an u64 but got the parsing error actual String not a u64.
@Pzixel
Couldn't we just call that on every field? I get it'd be slow, but if your bottleneck is parsing envars you probably have another problem