Apple Silicon ARM64 : Invalid access: Can not convert empty value.

[asmaan]

I have compiled poco for arm64 for Apple Silicon Native support. But for arm devices JSON parsing fails even for simplest json contracts. Please see below code snippet.

std::string json = "{\"clientConfig\" : \"Franky\"}";
        Poco::JSON::Parser jsonParser;
        Poco::Dynamic::Var jsonObject = jsonParser.parse(json);
        Poco::JSON::Object::Ptr jsonPtr = jsonObject.extract<Poco::JSON::Object::Ptr>();
        std::string testStr = jsonPtr->getValue<std::string>("clientConfig"); -> Here I get that exception.

If I check jsonPtr->get("clientConfig").isEmpty() , it returns true as well.

I tried to compile with -fvisibility=hidden as well but same result. If I compiled for x86_64 and run on Silicon Macs(through rosetta), it work fine. On iOS it works fine too. If I run in debug mode on Apple Silicon Mac, it works fine, issue is only in release mode.

Any ideas what's causing this?

[asmaan]

@obiltschnig Can you please take a quick look?

[obiltschnig]

I don't have an Apple Silicon Mac available. I can do a quick test on iOS maybe. I remember a similar issue on Android.

[obiltschnig]

Probably related: #2844

[asmaan]

Can you please guide me to android issue? I found few but they were not specific to this particular error. On iOS it works for me, issue is specific to ARM based Macs only. Another thing to add, If I run in debug mode, it works fine, issue is only in release mode.(Will put in description as well)

[asmaan]

@obiltschnig I looked at the android issue and it looks similar to that.But in my case exception is different. Any workarounds you can suggest?

[asmaan]

My suspicion is that this issue is under this snippet in Var.h where pHolder is nil

template <typename T>
	T convert() const
		/// Invoke this method to perform a safe conversion.
		///
		/// Example usage:
		///     Var any("42");
		///     int i = any.convert<int>();
		///
		/// Throws a RangeException if the value does not fit
		/// into the result variable.
		/// Throws a NotImplementedException if conversion is
		/// not available for the given type.
		/// Throws InvalidAccessException if Var is empty.
	{
		VarHolder* pHolder = content();

		if (!pHolder)
			throw InvalidAccessException("Can not convert empty value.");

		if (typeid(T) == pHolder->type()) return extract<T>();

		T result;
		pHolder->convert(result);
		return result;
	}

[asmaan]

Finally managed to figure it out. Set ENABLE_TESTABILITY to NO in your target's Release mode build settings . This will fix the issue. I don't know why poco needs this privilege to run. @obiltschnig Can you please help me understand why poco need this entitlement?

[obiltschnig]

Could you provide the detailed clang command line XCode uses in release build with both ENABLE_TESTABILITY set to on and off. It likely has to do with some compiler setting (-fvisibility maybe) that influence the way RTTI works across dylib boundaries.

[asmaan]

I can only see export ENABLE_TESTABILITY\=YES , no trace of this flag in clang command line. See below clan command.

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x objective-c++ -target arm64-apple-macos10.14 -fmessage-length=0 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit=0 -std=gnu++1z -stdlib=libc++ -fobjc-arc -fmodules -gmodules -fmodules-cache-path=/Users/testuser/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -fmodules-prune-interval=86400 -fmodules-prune-after=345600 -fbuild-session-file=/Users/testuser/Library/Developer/Xcode/DerivedData/ModuleCache.noindex/Session.modulevalidation -fmodules-validate-once-per-build-session -Wnon-modular-include-in-framework-module -Werror=non-modular-include-in-framework-module -Wno-trigraphs -fpascal-strings -Os -fno-common -Wno-missing-field-initializers -Wno-missing-prototypes -Werror=return-type -Wunreachable-code -Wno-implicit-atomic-properties -Werror=deprecated-objc-isa-usage -Wno-objc-interface-ivars -Werror=objc-root-class -Wno-arc-repeated-use-of-weak -Wimplicit-retain-self -Wno-non-virtual-dtor -Wno-overloaded-virtual -Wno-exit-time-destructors -Wduplicate-method-match -Wno-missing-braces -Wparentheses -Wswitch -Wunused-function -Wno-unused-label -Wno-unused-parameter -Wunused-variable -Wunused-value -Wempty-body -Wuninitialized -Wconditional-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wconstant-conversion -Wint-conversion -Wbool-conversion -Wenum-conversion -Wno-float-conversion -Wnon-literal-null-conversion -Wobjc-literal-conversion -Wshorten-64-to-32 -Wno-newline-eof -Wno-selector -Wno-strict-selector-match -Wundeclared-selector -Wdeprecated-implementations -Wno-c++11-extensions -DDEVELOPMENT_TEAM="TTTTTTTT" -Dtray=1 -DNS_BLOCK_ASSERTIONS=1 -DOBJC_OLD_DISPATCH_PROTOTYPES=0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fstrict-aliasing -Wprotocol -Wdeprecated-declarations -Winvalid-offsetof -g -Wno-sign-conversion -Winfinite-recursion -Wmove -Wcomma -Wblock-capture-autoreleasing -Wno-strict-prototypes -Wrange-loop-analysis -Wno-semicolon-before-method-body -iquote /

BuildProductsPath/Release -D_DARWIN_UNLIMITED_SELECT -DFD_SETSIZE=65536 -MMD -MT dependencies -MF

[obiltschnig]

Still no access to Apple Silicon Mac to reproduce. Moving to 1.13.

[matejk]

Latest version from devel works properly on Apple Silicon (M1 Pro Max) for the snippet provided by the submitter.

% c++ --version
Apple clang version 14.0.0 (clang-1400.0.29.202)
Target: arm64-apple-darwin22.3.0
Thread model: posix

% sw_vers 
ProductName:		macOS
ProductVersion:		13.2.1
BuildVersion:		22D68

[matejk]

However few other tests fail:

There were 3 errors: 
 1: N7CppUnit10TestCallerI8JSONTestEE.testValidJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1
 2: N7CppUnit10TestCallerI8JSONTestEE.testInvalidJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1
 3: N7CppUnit10TestCallerI8JSONTestEE.testInvalidUnicodeJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1

[bas524]

However few other tests fail:

There were 3 errors: 
 1: N7CppUnit10TestCallerI8JSONTestEE.testValidJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1
 2: N7CppUnit10TestCallerI8JSONTestEE.testInvalidJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1
 3: N7CppUnit10TestCallerI8JSONTestEE.testInvalidUnicodeJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1

It is not an error, please set environment variable POCO_BASE, which contains path to the poco source directory

[xzzh999]

in macos mini M1 arm64, i found the json function Array::Ptr Object::getArray always get empty array , and this test failed: it->second.type() == typeid(Array::Ptr) but i print the it->second.type().name() is eq typeid(Array::Ptr).name(), even add -frtti flag still not working. strange ...

i just fix this use:

Array::Ptr Object::getArray(const std::string& key) const
{
	ValueMap::const_iterator it = _values.find(key);

	// FIXME:
	// if ((it != _values.end()) && (it->second.type() == typeid(Array::Ptr)))
	if ((it != _values.end()) && (std::string(it->second.type().name()) == typeid(Array::Ptr).name()))
	{
		return it->second.extract<Array::Ptr>();
	}

	return 0;
}

[matejk]

I added the exact code as provided in the description to the unit test.

It works properly on M1 MacBook:

% c++ --version
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin23.1.0
Thread model: posix

% sw_vers
ProductName:		macOS
ProductVersion:		14.1.1
BuildVersion:		23B81

% uname -a
Darwin Matejs-MBP 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct  9 21:27:24 PDT 2023; root:xnu-10002.41.9~6/RELEASE_ARM64_T6000 arm64

And Ubuntu 22.04 running in Multipass on the same laptop:

$ c++ --version
c++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ lsb_release --all
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.3 LTS
Release:	22.04
Codename:	jammy

$ uname -a
Linux xts 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct  9 18:01:07 UTC 2 aarch64 aarch64 aarch64 GNU/Linux

[aleks-f]

@matejk please mark as fixed if it is verified

[matejk]

An additional unit tests was added in PR #4315 that covers issues reported in this ticket. The test passes on macOS running on Apple M1.

[matejk]

The problem can be reproduced on macOS with CMAKE if:

• CMAKE_BUILD_TYPE is Release
• set(CMAKE_CXX_VISIBILITY_PRESET hidden) is set

Poco 1.13 does not build on macOS with the aforementioned changes due to missing *_API directives on some classes and structs.

Poco does not build on Ubuntu Linux 22.04 (gcc 11.4) also with the same CMake settings.