glaze icon indicating copy to clipboard operation
glaze copied to clipboard

weird 'missing_key' bug since v4.0.3+

Open hs-vc opened this issue 10 months ago • 8 comments

Since Glaze version 4.0.3, the following minimal reproducible example produces an unexpected missing_key runtime error when parsing JSON.

This bug seems unusual because making seemingly unrelated modifications (e.g., small changes to the project structure) can prevent reproducing the issue.

src/main.cpp:

#include <glaze/glaze.hpp>

namespace {
    struct tmp_struct {
        bool success;
    };

    void parse_string(const std::string &m) {
        tmp_struct response{};

        auto e = glz::read<glz::opts{
            .error_on_unknown_keys = false,
            .error_on_missing_keys = true
        }>(response, m);

        if (e) {
            throw std::runtime_error(glz::format_error(e));
        };
    }
}


int main() {
    parse_string(R"({"success":true})");
    return 0;
}

src/unused.cpp:

#include <glaze/glaze.hpp>

namespace {
    struct tmp_struct {
        int retCode;
    };

    auto unused_function() {
        tmp_struct response{};
        return glz::read<glz::opts{
            .error_on_unknown_keys = false,
            .error_on_missing_keys = true
        }>(response, "");
    }
}

CMakeLists.txt:

cmake_minimum_required(VERSION 3.20)
project(glaze_test)

set(CMAKE_CXX_STANDARD 23)

include(FetchContent)

FetchContent_Declare(
        glaze
        GIT_REPOSITORY https://github.com/stephenberry/glaze.git
        GIT_TAG v5.0.0  # any version since 4.0.3 should produce same result
        GIT_SHALLOW TRUE
)

FetchContent_MakeAvailable(glaze)

# Do not change the order of sources
add_executable(glaze_test
        src/unused.cpp
        src/main.cpp
        )
target_link_libraries(glaze_test PRIVATE glaze::glaze)

This issue appears in v4.0.3 and later, including v5.0.0, but does not occur in earlier versions.

hs-vc avatar Mar 06 '25 08:03 hs-vc

What compiler are you using? And, if you remove the anonymous namespace (namespace {) does the issue go away?

stephenberry avatar Mar 06 '25 20:03 stephenberry

This issue was found on GCC 14.2.0, but it works fine with GCC 13.3.0. The OS in use is Ubuntu 24.04.2 LTS.

Regarding the namespace issue, removing the anonymous namespace in either src/unused.cpp or src/main.cpp resolves the issue, but removing it from both files does not.

hs-vc avatar Mar 07 '25 02:03 hs-vc

After further checking the output of each object file, I found they contain a conflicting symbol. Running nm unused.cpp.o | grep ZN3glz10comparitorIL shows:

0000000000000000 W _ZN3glz10comparitorIL_ZZNS_12decode_indexIXtlNS_4optsELj10ELb1ELb0ELb0ELb1ELb1ELb0ELb0ELc32ELh3ELb1ELb0ELb1EEEN12_GLOBAL__N_110tmp_structELm0ERS4_JRmERNS_7contextERPKcSB_EEvOT2_OT4_OT5_OT6_DpOT3_E15KeyWithEndQuoteELm8EcEEbPKT1_

And running nm main.cpp.o | grep ZN3glz10comparitorIL shows the exact same symbol.

The demangled name of the conflicting symbol is:

bool glz::comparitor<glz::decode_index<
    glz::opts{10u, true, false, false, true, true, false, false, (char)32, (unsigned char)3, true, false, true}, 
    (anonymous namespace)::tmp_struct, 
    0ul, 
    (anonymous namespace)::tmp_struct&, 
    unsigned long&, 
    glz::context&, 
    char const*&, 
    char const*&
>((anonymous namespace)::tmp_struct&, glz::context&, char const*&, char const*&, unsigned long&)::KeyWithEndQuote, 8ul, char>(char const*)

What I don’t understand is why this symbol is marked as globally visible with a weak (W) linkage, even though it is instantiated with a struct inside an anonymous namespace. While the exact reason for this is not clear, it seems that at link time, the definition from unused.cpp.o is being selected instead of the one from main.cpp.o.

hs-vc avatar Mar 07 '25 06:03 hs-vc

It seems using const std::string_view& as template argument result in some unexpected behavior, as in below minimal project.

src/a.hpp:

#pragma once
#include <string_view>

template <const std::string_view& Str>
auto get_length() {
    return Str.length();
}

src/main.cpp:

#include <iostream>
#include "a.hpp"

namespace {
    static constexpr std::string_view sssss = "123";
}


int main() {
    std::cout << get_length<sssss>() << std::endl;
    return 0;
}

src/unused.cpp:

#include <string_view>
#include "a.hpp"


namespace {
    static constexpr std::string_view sssss = "12345";
    auto unused_function() {
        return get_length<sssss>();
    }
}

CMakeLists.txt:

cmake_minimum_required(VERSION 3.30)
project(linkage_test)

set(CMAKE_CXX_STANDARD 23)

add_executable(linkage_test src/unused.cpp src/main.cpp)

Surprisingly, this program outputs 5, not 3. The issue arises because unused.cpp.o and main.cpp.o contain conflicting definitions of auto get_length<(anonymous namespace)::sssss>(), as shown by running nm:

nm unused.cpp.o main.cpp.o | grep _Z10get_lengthIL_ZN12_GLOBAL__N_1L5sssssEEEDav 0000000000000000 W _Z10get_lengthIL_ZN12_GLOBAL__N_1L5sssssEEEDav 0000000000000000 W _Z10get_lengthIL_ZN12_GLOBAL__N_1L5sssssEEEDav

The function is emitted as a W, meaning the linker arbitrarily picks one definition, which results in unexpected behavior.

hs-vc avatar Mar 07 '25 08:03 hs-vc

Wow, thanks for diving into this issue! This is very interesting. The template instantiation should be based on the reference of the std::string_view, which should have different addresses. This seems like a compiler bug. Do you want me to report this to GCC bugzilla? I can then add their feedback here.

stephenberry avatar Mar 07 '25 15:03 stephenberry

Right, this is clearly a GCC 14.2.0 issue, given that GCC 13 works correctly. I will report this on GCC Bugzlilla and share here.

hs-vc avatar Mar 09 '25 15:03 hs-vc

I have reported this to the GCC team here: GCC Bug 119194.

hs-vc avatar Mar 11 '25 13:03 hs-vc

Excellent, thanks!

stephenberry avatar Mar 11 '25 13:03 stephenberry

GCC 14.3 was released (May 23, 2025) with a fix for this.

Fixed for 14.3/15.

So, I'm closing this issue, but reopen if this somehow shows up again in later versions of GCC.

stephenberry avatar May 30 '25 10:05 stephenberry