C++17 std::optional feature not enabled

[Stevan1996]

Description

I tried to use std::optional with the library. I am using CMake, and, in my CMakeLists.txt, I set the language standard to C++23 with set(CMAKE_CXX_STANDARD 23).

When trying to read a JSON field as std::optional<std::string>>, I get a compilation error that no matching overload could be found. By skimming over the source code, I could not find any checks for JSON_HAS_CPP_17 in the implementation of the get method.

Reproduction steps

Reading a JSON field as std::optional<T> with json.get<std::optional<T>>.

Expected vs. actual results

Expected The get method should be able to return std::optional<T>.

Actual Compiler cannot find a matching overload when using std::optional<T> as template parameter of get.

Minimal code example

nlohmann::json json;
json.get<std::optional<std::string>>();

Error messages

error C2672: 'nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get': no matching overloaded function found

Compiler and operating system

MSVC 19.44.35213.0 & Windows 11

Library version

3.12.0

Validation

• [ ] The bug also occurs if the latest version from the develop branch is used.
• [x] #4865

[gregmarr]

Are you using JSON_USE_IMPLICIT_CONVERSIONS set as false? If so, you can't use 3.12.0, you need to use the latest develop branch until a new release is made.

By skimming over the source code, I could not find any checks for JSON_HAS_CPP_17 in the implementation of the get method.

That's correct, it isn't part of get, it's done through a from_json function.

[Stevan1996]

I did not set JSON_USE_IMPLICIT_CONVERSIONS. According to the documentation, the default value should be true. I explicitly set the variable with set(JSON_ImplicitConversions ON) in the CMakeLists.txt, and it still does not work.

[gregmarr]

It works in general: https://www.godbolt.org/z/1qhe8PY55

Can you try to compile that simple file in your environment and share the full error? There may be clues in that output.

[Stevan1996]

With a minimal example project, I still get the error that a matching overload cannot be found.

The output of the build process:

====================[ Build | json_optional | Debug ]===========================
"C:\Program Files\CMake\bin\cmake.exe" --build C:\json_optional_example\cmake-build-debug --target json_optional -j 22
[1/4] Scanning C:\json_optional_example\main.cpp for CXX dependencies
[2/4] Generating CXX dyndep file CMakeFiles\json_optional.dir\CXX.dd
[3/4] Building CXX object CMakeFiles\json_optional.dir\main.cpp.obj
FAILED: CMakeFiles/json_optional.dir/main.cpp.obj 
C:\PROGRA~1\MIB055~1\2022\ENTERP~1\VC\Tools\MSVC\1444~1.352\bin\Hostx64\x64\cl.exe  /nologo /TP  -external:IC:\vcpkg\installed\x64-windows\include -external:W0 /DWIN32 /D_WINDOWS /EHsc /Ob0 /Od /RTC1 -std:c++latest -MDd -Zi /showIncludes @CMakeFiles\json_optional.dir\main.cpp.obj.modmap /FoCMakeFiles\json_optional.dir\main.cpp.obj /FdCMakeFiles\json_optional.dir\ /FS -c C:\json_optional_example\main.cpp
C:\json_optional_example\main.cpp(5): error C2672: 'nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get': no matching overloaded function found
C:\vcpkg\installed\x64-windows\include\nlohmann/json.hpp(1812): note: could be 'unknown-type nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get(void) noexcept'
C:\json_optional_example\main.cpp(5): note: 'unknown-type nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get(void) noexcept': could not deduce template argument for '__formal'
C:\vcpkg\installed\x64-windows\include\nlohmann/json.hpp(1811): note: 'type': is not a member of any direct or indirect base class of 'std::enable_if<false,int>'
C:\vcpkg\installed\x64-windows\include\nlohmann/json.hpp(1771): note: or       'unknown-type nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get(void) noexcept(<expr>) const'
C:\json_optional_example\main.cpp(5): note: Failed to specialize function template 'unknown-type nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get(void) noexcept(<expr>) const'
C:\json_optional_example\main.cpp(5): note: With the following template arguments:
C:\json_optional_example\main.cpp(5): note: 'ValueTypeCV=std::optional<std::string>'
C:\json_optional_example\main.cpp(5): note: 'ValueType=std::optional<std::string>'
C:\vcpkg\installed\x64-windows\include\nlohmann/json.hpp(1773): note: 'nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get_impl': no matching overloaded function found
C:\vcpkg\installed\x64-windows\include\nlohmann/json.hpp(1736): note: could be 'unknown-type nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get_impl(nlohmann::json_abi_v3_12_0::detail::priority_tag<4>) noexcept const'
C:\vcpkg\installed\x64-windows\include\nlohmann/json.hpp(1723): note: or       'nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void> nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get_impl(nlohmann::json_abi_v3_12_0::detail::priority_tag<3>) const'
C:\vcpkg\installed\x64-windows\include\nlohmann/json.hpp(1700): note: or       'BasicJsonType nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get_impl(nlohmann::json_abi_v3_12_0::detail::priority_tag<2>) const'
C:\vcpkg\installed\x64-windows\include\nlohmann/json.hpp(1675): note: or       'ValueType nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get_impl(nlohmann::json_abi_v3_12_0::detail::priority_tag<1>) noexcept(<expr>) const'
C:\vcpkg\installed\x64-windows\include\nlohmann/json.hpp(1633): note: or       'ValueType nlohmann::json_abi_v3_12_0::basic_json<std::map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::json_abi_v3_12_0::adl_serializer,std::vector<uint8_t,std::allocator<uint8_t>>,void>::get_impl(nlohmann::json_abi_v3_12_0::detail::priority_tag<0>) noexcept(<expr>) const'
ninja: build stopped: subcommand failed.

The code I used can be found here: https://github.com/Stevan1996/json_optional_example

[pierr3]

Also facing the same on clang-16-apple arm64, using the same minimal example as above. Included via VCPKG and JSON_ImplicitConversions defined as ON in CMake.

[RogerSanders]

I just lost three hours to this thinking the problem was elsewhere, so to clarify for everyone:

1. Using std::optional works on the current development head
2. Using std::optional is broken in the current latest 3.12.0 release

The reason it's broken is because in the single-include release, at line 2945, we have this code:

#ifndef JSON_USE_IMPLICIT_CONVERSIONS
    #define JSON_USE_IMPLICIT_CONVERSIONS 1
#endif

And then after this at line 4835 we have this:

#ifndef JSON_USE_IMPLICIT_CONVERSIONS
template<typename BasicJsonType, typename T>
void from_json(const BasicJsonType& j, std::optional<T>& opt)
{
    if (j.is_null())
    {
        opt = std::nullopt;
    }
    else
    {
        opt.emplace(j.template get<T>());
    }
}
#endif // JSON_USE_IMPLICIT_CONVERSIONS

Hence the std::optional from_json overload is always excluded by the preprocessor. Issue was fixed in #4742 four months ago, but no release since then, so it's still broken on the latest "blessed" release version. Using the latest main or patching the header to remove the #ifndef JSON_USE_IMPLICIT_CONVERSIONS guard fixes the issue

[RogerSanders]

Also, beware another trap - due to a similar line ordering problem, you must add #include <optional> before including json.hpp, or you'll get another compilation issue, as the current release version tries to include optional before building the JSON_HAS_CPP_17 define, so it never includes it, then assumes it has already been included later once you fix the above from_json1 issue. This is also fixed in main.

[nlohmann]

@RogerSanders I'm sorry that the bug wasted your time :-(. I will try to make a 3.12.1 release soon.

[RogerSanders]

No worries, bugs happen. Just ended up here after chasing the problem down, so figured I'd leave a solution for others who end up here too. I'm actually still chasing what might be another issue even on main TBH. I'm trying to adjust existing code that was using the "NLOHMANN_JSONIFY_ALL_THINGS" workaround from here: https://www.kdab.com/jsonify-with-nlohmann-json/ To just use vanilla nlohmann now that std::optional support is in. Still having an issue using existing types with NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE instead I'm trying to debug.

EDIT: Problem was on my end. Current master works great, as long as I use NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT to replace NLOHMANN_JSONIFY_ALL_THINGS for my use case. Using NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE only works if the optional members are explicitly included in the json as null, IE, someValue: null, while they are omitted from the json in my case, so I need the default fallback for conversion to be happy. This is consistent and makes perfect sense.

[MegaMech]

Please also be sorry that the bug wasted my time as well. :P

Thank you for working on a 3.12.1 soon. Appreciate it!

[dsieradzki]

Hi, I’m also having an issue with optional. I used develop, but it’s not working. Am I doing something wrong, or could this be another problem? It does work in the case without struct, though.

    nlohmann::json json;
    json.get<std::optional<std::string>>();

https://www.godbolt.org/z/W6918z9Tc

#include <string>
#include "nlohmann/json.hpp"

struct Simple {
    std::optional<std::string> field;
    NLOHMANN_DEFINE_TYPE_INTRUSIVE(Simple, field)
};

int main()
{

    auto res = nlohmann::json::parse("{\"field\":\"\"}");
    auto obj1 = res.get<Simple>();

    // Not working - should be std::nullopt
    res = nlohmann::json::parse("{}");
    auto obj2 = res.get<Simple>();
}

[pierr3]

Use NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT and see @RogerSanders's post above:

EDIT: Problem was on my end. Current master works great, as long as I use NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT to replace NLOHMANN_JSONIFY_ALL_THINGS for my use case. Using NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE only works if the optional members are explicitly included in the json as null, IE, someValue: null, while they are omitted from the json in my case, so I need the default fallback for conversion to be happy. This is consistent and makes perfect sense.

On another note, PR with patch submitted for vcpkg, will keep an eye out to also update when the new version comes in - thanks for working on this!

[MegaMech]

Note to remove JSON_USE_IMPLICIT_CONVERSIONS if you updated to the latest vcpkg

[ansible42]

    struct TagCfgStructBase
    {
        std::string Name = "";
        std::optional<std::string> Units;
    };

    NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT(TagCfgStructBase, Name, Units)

    TEST_CASE("TestCfgStructScaled serialization & deserialization")
    {
        TagCfgStructBase tag_without_units = {.Name = "Unscaled"};
        nlohmann::json json_tag_without_units = tag_without_units;
        std::cout << json_tag_without_units.dump() << "\n";
        REQUIRE(json_tag_without_units.dump().find("Units") == std::string::npos);
    }

So this producing {"Name":"Unscaled","Units":null} is expected behavior?

[Su5eD]

I might be running into the same issue here, using develop commit a0e9fb1e638cfbb5b8b556b7c51eaa81977bad48:

std::optional<std::string> value = std::nullopt;
nlohmann::json json = value;

nlohmann::json parsed = nlohmann::json::parse(json.dump());
std::optional<std::string> newVal = parsed; // ERROR

Results in

[json.exception.type_error.302] type must be string, but is null

I would expect newVal to be a std::nullopt after deserialization.

[gregmarr]

First, if possible, you should set this before including the header to disable the implicit conversions:

#define JSON_USE_IMPLICIT_CONVERSIONS 0

Then get the data from the json object like this:

  auto newVal = parsed.get<std::optional<std::string>>();

and then it will properly be an unengaged optional when you are reading the json.dump(), which is just null. https://www.godbolt.org/z/Kxb5vPM7a