Need more nuanced skipping for writing

[DimichGD]

Hello. I have an array of json objects where some of them have an additional field. I wrote a simple test and got same error.

#include <iostream>
#include <variant>
#include "glaze/json/read.hpp"

struct Timing1
{
	int x;
	int y;
};

template<>
struct glz::meta<Timing1>
{
	using T = Timing1;
	static constexpr auto value = object("x", &T::x, "y", &T::y);
};

struct Timing2
{
	int y;
};

template<>
struct glz::meta<Timing2>
{
	using T = Timing2;
	static constexpr auto value = object("y", &T::y);
};

int main()
{
	std::string json_text = R"({"y":7})";
	std::variant<Timing1, Timing2> var;
	auto err = glz::read_json(var, json_text);
	if (err)
		std::cout << glz::format_error(err) << std::endl;

	return 0;
}

[stephenberry]

Take a look at the Variant Handling documentation.

For auto-education to work there must be uniqueness between the keys. How do we know if y refers to Timing1 or Timing2, because both objects contain y? This is simply a logical issue, that the variant type cannot be deduced without additional information. If you provided x, we would know it must be Timing1.

The solution is to use tagged variants for cases like these. Where you provide a tag key name and a list of type names associated with each type.

Example:

using tagged_variant = std::variant<put_action, delete_action>;

template <>
struct glz::meta<tagged_variant>
{
   static constexpr std::string_view tag = "action";
   static constexpr auto ids = std::array{"PUT", "DELETE"}; //Defaults to glz::name_v of the type if ids is not supplied
};

[DimichGD]

I don't have any tags. I guess I can't use variant. Objects look like this:

struct Timing
{
	int conditions; // this field may not exist
	std::array<int, 4> flashColor;
	int flashDuration;
	int flashScope;
	int frame;
	std::optional<Sound> se; // this field can be 'null'
};

Changing conditions member from int to std::optional<int> solves reading issue. But I don't know how to skip it on writing. Can I achieve this with custom serialization?

[stephenberry]

Glaze will skip writing null optionals by default. So if you use std::optional<int> conditions; then it won't be written if null. You can always set it to null to disable output.

However, do you need to ensure that conditions is never written at all, even when it has data? Usually data that cannot round-trip is a design flaw, so if you could help me understand the use case it would help develop a solution.

I'm working on selective compile time operations based on key names. This would enable even finer control over skipping, but has not yet been implemented.

[DimichGD]

For example:

{
	"conditions": 0,
	"flashColor": [ 255, 255, 221, 119 ],
	"flashDuration": 5,
	"flashScope": 1,
	"frame": 0,
	"se": null
},
{
	"flashColor": [ 255, 255, 255, 255 ],
	"flashDuration": 5,
	"flashScope": 0,
	"frame": 2,
	"se": { "name": "Blow1", "pan": 0, "pitch": 100, "volume": 90 }
},

se can be null, so I can't use skip_null_members for writing. Right now I come up with this solution:

template <>
struct glz::to<glz::JSON, Timing>
{
	template <auto Opts, typename T>
	static void write_kv(std::string_view key, T& value, bool last, is_context auto&& ctx, auto&& b, auto&& ix)
	{
		serialize<JSON>::op<Opts>(key, ctx, b, ix);
		dump<':'>(b, ix);
		serialize<JSON>::op<Opts>(value, ctx, b, ix);
		if (!last)
			dump<','>(b, ix);
	}

	template <auto Opts>
	static void op(Timing& value, is_context auto&& ctx, auto&& b, auto&& ix)
	{
		dump<'{'>(b, ix);
		if (value.conditions.has_value())
			write_kv<Opts>("conditions", value.conditions.value(), false, ctx, b, ix);

		write_kv<Opts>("flashColor", value.flashColor, false, ctx, b, ix);
		write_kv<Opts>("flashDuration", value.flashDuration, false, ctx, b, ix);
		write_kv<Opts>("flashScope", value.flashScope, false, ctx, b, ix);
		write_kv<Opts>("frame", value.frame, false, ctx, b, ix);
		write_kv<Opts>("se", value.se, true, ctx, b, ix);

		dump<'}'>(b, ix);
	}
};

It works. But it looks kinda ugly.

[stephenberry]

Thanks for sharing. I'm glad you found at least a temporary solution, but I'd like to offer more flexibility here. I'll consider this case as I look into compile time configuration functions that I'm developing for glz::meta.

[stephenberry]

Note: glz::skip{} works with glz::custom for reading, but doesn't not currently work for writing, because this is a hard problem to solve. But, this is another approach that can be useful in some cases.

[stephenberry]

Glaze recently got skip_if support for glz::meta: #2029

More documentation here: https://stephenberry.github.io/glaze/skip-keys/?h=skip_if#combining-skip-and-skip_if

This offers the nuanced skipping that you wanted:

struct Timing {
    std::optional<int> conditions;
    std::optional<Sound> se;
};

template <>
struct glz::meta<Timing> {
    template <class T>
    static constexpr bool skip_if(T&& value, std::string_view key, const glz::meta_context&) {
        // Skip 'conditions' if it is empty (null)
        if (key == "conditions") {
            if constexpr (requires { value.has_value(); }) {
                return !value.has_value();
            }
        }
        return false;
    }
};

// Enable writing nulls globally so 'se' is written as null, 
// while 'conditions' is skipped by our custom skip_if logic.
constexpr auto opts = glz::opts{.skip_null_members = false};
glz::write<opts>(obj, buffer);