Copying/pickling regression in 0.14

[bocklund]

In PyCalphad, we have some tests that test deepcopying/pickling and our tests have been showing a regression for SymEngine 0.14 on Windows only across Python versions.

I haven't been able to come up with a pure SymEngine reproducer yet, but I've made some notes in an issue in the PyCalphad repo: https://github.com/pycalphad/pycalphad/issues/601 if anyone has ideas on changes that could have caused this or what I can do to make a pure SymEngine reproducible example.

[isuruf]

There were changes to the pickling in the C++ library. If you can make a pure SymEngine reproducible example I can have a look

[bocklund]

I've been trying some today, but haven't been able to create a pure SymEngine reproducer.

Between 0.13 and 0.14, commit 50310f008 in the C++ library changed symengine/serialize-cereal.h such that RCPBasicAwareOutputArchive.save_rcp_basic saves memory addresses and then loads objects by memory address in RCPBasicAwareInputArchive.load_rcp_basic.

Could it be that memory addresses as identifiers and the large expressions we have in PyCalphad ends up re-using or somehow corrupting memory address-based deserialization? That would explain some of the non-determinism we are seeing.

[bjodah]

@bocklund could it be that there's another .so object other than symengine.py that's loaded in your python process, and that also uses cereal (possibly another version thereof)? I was just remembering this comment from a while back. This is a far shot though, since I think we would be seeing issues both before and after the symengine commit you've identified if this was the culprit.

[bocklund]

Not that I know of, but it's not impossible. We've been pinned in PyCalphad to SymEngine==0.13 and it's been totally stable and several different people and machines can reproduce (with PyCalphad) on SymEngine >=0.14 (Windows only).

[bjodah]

When building SymEngine with gcc-14 I see some warnings that made me think of this issue again:

[15/78] Building CXX object symengine/tests/basic/CMakeFiles/test_serialize-cereal.dir/test_serialize-cereal.cpp.o
In file included from /usr/include/c++/14/bits/shared_ptr_atomic.h:33,
                 from /usr/include/c++/14/memory:81,
                 from /work/external/symengine/symengine/utilities/catch/catch.hpp:2948,
                 from /work/external/symengine/symengine/tests/basic/test_serialize-cereal.cpp:1:
In member function ‘std::__atomic_base<_IntTp>::__int_type std::__atomic_base<_IntTp>::fetch_add(__int_type, std::memory_order) [with _ITp = unsigned int]’,
    inlined from ‘std::__atomic_base<_IntTp>::__int_type std::__atomic_base<_IntTp>::operator++(int) [with _ITp = unsigned int]’ at /usr/include/c++/14/bits/atomic_base.h:382:25,
    inlined from ‘SymEngine::RCP<T>::RCP(T*) [with T = const SymEngine::Integer]’ at /work/external/symengine/symengine/symengine_rcp.h:117:26,
    inlined from ‘SymEngine::RCP<T> SymEngine::rcp_static_cast(const RCP<T1>&) [with T2 = const Integer; T1 = const Basic]’ at /work/external/symengine/symengine/symengine_rcp.h:225:25,
    inlined from ‘SymEngine::RCP<const T2> SymEngine::RCPBasicAwareInputArchive<Archive>::load_rcp_basic() [with T = SymEngine::Integer; Archive = cereal::BinaryInputArchive]’ at /work/external/symengine/symengine/type_codes.inc:1:1:
/usr/include/c++/14/bits/atomic_base.h:631:34: warning: ‘unsigned int __atomic_fetch_add_4(volatile void*, unsigned int, int)’ writing 4 bytes into a region of size 0 overflows the destination [-Wstringop-overflow=]
  631 |       { return __atomic_fetch_add(&_M_i, __i, int(__m)); }
      |                ~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
In member function ‘SymEngine::RCP<const T2> SymEngine::RCPBasicAwareInputArchive<Archive>::load_rcp_basic() [with T = SymEngine::Integer; Archive = cereal::BinaryInputArchive]’:
cc1plus: note: destination object is likely at address zero
In member function ‘std::__atomic_base<_IntTp>::__int_type std::__atomic_base<_IntTp>::fetch_add(__int_type, std::memory_order) [with _ITp = unsigned int]’,
    inlined from ‘std::__atomic_base<_IntTp>::__int_type std::__atomic_base<_IntTp>::operator++(int) [with _ITp = unsigned int]’ at /usr/include/c++/14/bits/atomic_base.h:382:25,
    inlined from ‘SymEngine::RCP<T>::RCP(T*) [with T = const SymEngine::Integer]’ at /work/external/symengine/symengine/symengine_rcp.h:117:26,
    inlined from ‘SymEngine::RCP<T> SymEngine::rcp_static_cast(const RCP<T1>&) [with T2 = const Integer; T1 = const Basic]’ at /work/external/symengine/symengine/symengine_rcp.h:225:25,
    inlined from ‘SymEngine::RCP<const T2> SymEngine::RCPBasicAwareInputArchive<Archive>::load_rcp_basic() [with T = SymEngine::Integer; Archive = cereal::BinaryInputArchive]’ at /work/external/symengine/symengine/type_codes.inc:1:1:
/usr/include/c++/14/bits/atomic_base.h:631:34: warning: ‘unsigned int __atomic_fetch_add_4(volatile void*, unsigned int, int)’ writing 4 bytes into a region of size 0 overflows the destination [-Wstringop-overflow=]
  631 |       { return __atomic_fetch_add(&_M_i, __i, int(__m)); }
      |                ~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
In member function ‘SymEngine::RCP<const T2> SymEngine::RCPBasicAwareInputArchive<Archive>::load_rcp_basic() [with T = SymEngine::Integer; Archive = cereal::BinaryInputArchive]’:
cc1plus: note: destination object is likely at address zero

I haven't figured out if this warning is spurious or not.