Allow map/transform JSON fields into struct fields via projection function

[leha-bot]

Current glz API seems doesn't support JSON field name customizations like one of these cases:

• JSON fields are in camelCase, and struct fields are in snake case (this case sometimes may appear while re-styling JSON APIs, but internal API should be preserved), e.g.:

{
"languageId": "cpp",
"version": 0,
"uri": "https://localhost:8888/1.cpp"
}

struct file {
 std::string_view language_id;
 std::string_view uri;
 int version;
}

In this case glz would apply some transform function "to_snake" which "projects" the json field "languageId" to another string literal "language_id" and map it into file::language_id. It would be good to define the consteval(constexpr?) function with string fields names projection like:

consteval auto to_snake(const auto in_field_name) {
  // Parse magic with case convertation...
 return result;
}

// Potential api may looks like:
auto file = glz::read_json<struct file, project_field_names_using<to_snake>>(in_json);

• JSON contains C/C++ reserved identifiers, and we have to map into whole family of structures with appended '_':

{
  "class": {
   "name": "myclass",
    "fields": [],
    "methods": []
   },
   "location": {
     "file": "myclass.h",
     "line": 16
   }
}

struct class_identifier_info {
  location location_;
  class_info class_;
}

For this case it may compare the input JSON keys with reserved C++ identifiers:

auto info = glz::read_json<class_identifier_info, project_field_names_using<append_underscore_to_cpp_identifiers>>(in_json);

• Some location APIs which uses the {"latitude", "longitude"} pair with distinct names in their JSON API:

{
  "lat": "47,15 N",
  "lon": "56,4 W"
}

vs

{
  "latitude": "47,15 N",
  "longitude": "56,4 W"
}

auto location = glz::read_json<location, project_field_names_using<adapt_lat_lon_names>(in_json);

The reverse operation can also be projected from reflected field name to JSON field name.

The glz::meta API may also looks like:

template <>
struct glz::meta<some_type> {
  static constexpr auto project_field_name_from(auto json_field_name) {
     // default is simply identity function which returns same json_field_name
    return json_field_name;
  }
  static constexpr auto project_field_name_to(auto native_field_name) {
     // default is simply identity function which returns same native_field_name
    return native_field_name;
  }
};

These proposed operations are conceptually same as projections in C++ stdlib ranged algorithms, but for JSON field names <-> native struct field names.

Sorry if describe it vaguely, and thanks for your lib.

[stephenberry]

The glz::meta API allows you to rename your fields however you like. If you look at the README it will explain a bit.

Here's an example:

template <>
struct glz::meta<file>
{
  using T = file;
  static constexpr auto value = object(“languageId”, &T::language_id, &T::uri, &T::version);
};

This will parse languageId in camel case into the C++ variable name language_id. You can customize any of your fields in this manner.

Does this support your needs?

[leha-bot]

I thought about it, but don't exactly know, is it possible to generalize to all identifiers, and another question: will another fields be read from JSON and deserialized into C++ struct?

And I see an issue with scalability - we have to (re)write all fields likewise instead of transform() logic.

I also looked into the code and began to work on my issue locally, I could try to make a PR with this feature, if you don't mind.

[stephenberry]

I agree with complexities in scalability and transform logic would be cleaner. This is achievable at compile time, but would be somewhat complicated. The transformation would have to happen when populating the glz::reflect struct in glaze/core/reflect.hpp. You could try making a PR for this, and I can help you walk through challenges. But, it might be more efficient for me to figure out a good plan of attack first. Yet, go for it if you like.

One tricky aspect is that these lambda functions like project_field_name_from need to allocate transformed strings at compile time. We can use std::array within these constexpr functions, but they would need to be templated on the number of keys and the maximum length key.

One last thought: I'm trying to avoid making top level APIs that are esoteric and complex, when we have reflection in C++26 coming soon. I don't want to add features that will be superseded by C++26 reflection, but rather add features that will complement it. I feel like this transformation might be better achieved with the coming reflection features since the transformation logic has to be implemented at the top level.

[leha-bot]

I think that we can "be inspired" by reflect-cpp's rfl::Rename implementation. The example of their API could be viewed here: https://github.com/getml/reflect-cpp?tab=readme-ov-file#more-comprehensive-example They also called it "processors": https://github.com/getml/reflect-cpp/blob/main/docs/concepts/processors.md

[stephenberry]

@leha-bot, I finally got around to thinking about this (active pull request: #1782). The current approach is like your suggestion and quite straightforward, and it works at compile time, which is critical.

Currently this solution doesn't provide different projected name changes for reading and writing like your example. If this is needed in the future, support could be added, but it would be more complex under the hood. For now, I figured support for a straightforward name change is sufficient for most needs. Let me know your thoughts.

Example:

struct renamed_t
{
   std::string first_name{};
   std::string last_name{};
   int age{};
};

template <>
struct glz::meta<renamed_t>
{
   static constexpr std::string_view rename_key(const std::string_view key) {
      if (key == "first_name") {
         return "firstName";
      }
      else if (key == "last_name") {
         return "lastName";
      }
      return key;
   }
};

suite rename_tests = [] {
   "rename"_test = [] {
      renamed_t obj{};
      std::string buffer{};
      expect(not glz::write_json(obj, buffer));
      expect(buffer == R"({"firstName":"","lastName":"","age":0})") << buffer;
      
      buffer = R"({"firstName":"Kira","lastName":"Song","age":29})";
      
      expect(not glz::read_json(obj, buffer));
      expect(obj.first_name == "Kira");
      expect(obj.last_name == "Song");
      expect(obj.age == 29);
   };
};

[stephenberry]

Support for key transformation with dynamic memory std::string at compile time has been added as well:

struct suffixed_keys_t
{
   std::string first{};
   std::string last{};
};

template <>
struct glz::meta<suffixed_keys_t>
{
   static constexpr std::string rename_key(const auto key) {
      return std::string(key) + "_name";
   }
};

"suffixed keys"_test = [] {
  suffixed_keys_t obj{};
  std::string buffer{};
  expect(not glz::write_json(obj, buffer));
  expect(buffer == R"({"first_name":"","last_name":""})") << buffer;
  
  buffer = R"({"first_name":"Kira","last_name":"Song"})";
  
  expect(not glz::read_json(obj, buffer));
  expect(obj.first == "Kira");
  expect(obj.last == "Song");
};