gz-sim icon indicating copy to clipboard operation
gz-sim copied to clipboard

:farmer: Homebrew test regressions and segfaults in mac.monterey

Open Crola1702 opened this issue 10 months ago • 1 comments

Environment

  • OS Version: Mac Monterey
  • Source gz-sim main homebrew

Description

  • Expected behavior: Tests should pass
  • Actual behavior: 2 CmdLine tests and 2 Integration tests are failing

Steps to reproduce

  1. Run a build in https://build.osrfoundation.org/job/ignition_gazebo-ci-main-homebrew-amd64/
  2. See tests fail

Output

Reference build: https://build.osrfoundation.org/job/ignition_gazebo-ci-main-homebrew-amd64/198/

Test regressions:

CmdLine Cache Fuel World test output:
63: [ RUN      ] CmdLine.CachedFuelWorld
63: Running command [/usr/local/opt/ruby/bin/ruby /usr/local/bin/gz sim -s  -r -v 4 --iterations 5 https://fuel.gazebosim.org/1.0/openroboticstest/worlds/test%20world]
63: sh: line 1: 68765 Segmentation fault: 11  /usr/local/opt/ruby/bin/ruby /usr/local/bin/gz sim -s -r -v 4 --iterations 5 https://fuel.gazebosim.org/1.0/openroboticstest/worlds/test%20world 2>&1
63: /Users/jenkins/jenkins-agent/workspace/ignition_gazebo-ci-main-homebrew-amd64/ign-gazebo/src/gz_TEST.cc:113: Failure
63: Expected: (output.find("Cached world found.")) != (std::string::npos), actual: 18446744073709551615 vs 18446744073709551615
63: [Wrn] [LocalCache.cc:165] Server directory does not exist [/Users/jenkins/jenkins-agent/workspace/ignition_gazebo-ci-main-homebrew-amd64/ign-gazebo/test/worlds/fuel.ignitionrobotics.org]
63: Stack trace (most recent call last):
63: #8    Object "libdyld.dylib", at 0x7ff807df8375, in dyld4::LibSystemHelpers::exit(int) const + 11
63: #7    Object "libsystem_c.dylib", at 0x7ff807ce4bfe, in exit + 35
63: #6    Object "libsystem_c.dylib", at 0x7ff807ce4de4, in __cxa_finalize_ranges + 409
63: #5    Object "libdart.6.13.0.dylib", at 0x120ce584a, in std::__1::__hash_table<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::__unordered_map_hasher<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::__unordered_map_equal<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> > > >::~__hash_table() + 18
63: #4    Object "libdart.6.13.0.dylib", at 0x120ce5898, in std::__1::__hash_table<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::__unordered_map_hasher<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::__unordered_map_equal<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> > > >::__deallocate_node(std::__1::__hash_node_base<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, void*>*>*) + 48
63: #3    Object "libsystem_platform.dylib", at 0x7ff807e01dfc, in _sigtramp + 28
63: #2    Object "libgz-tools2-backward.dylib", at 0x105de1b1b, in backward::SignalHandling::sig_handler(int, __siginfo*, void*) + 13
63: #1    Object "libgz-tools2-backward.dylib", at 0x105de1b89, in backward::SignalHandling::handleSignal(int, __siginfo*, void*) + 79
63: #0    Object "libgz-tools2-backward.dylib", at 0x105de1d17, in backward::StackTraceImpl<backward::system_tag::darwin_tag>::load_from(void*, unsigned long, void*, void*) + 33
63: 
63: [  FAILED  ] CmdLine.CachedFuelWorld (2831 ms)
CmdLine Random Seed Value output:
63: [ RUN      ] CmdLine.RandomSeedValue
63: Running command [/usr/local/opt/ruby/bin/ruby /usr/local/bin/gz sim -s  -r -v 4 --seed 5 --iterations 5]
63: sh: line 1: 68769 Segmentation fault: 11  /usr/local/opt/ruby/bin/ruby /usr/local/bin/gz sim -s -r -v 4 --seed 5 --iterations 5 2>&1
63: /Users/jenkins/jenkins-agent/workspace/ignition_gazebo-ci-main-homebrew-amd64/ign-gazebo/src/gz_TEST.cc:124: Failure
63: Expected: (output.find("Setting seed value")) != (std::string::npos), actual: 18446744073709551615 vs 18446744073709551615
63: Stack trace (most recent call last):
63: #8    Object "libdyld.dylib", at 0x7ff807df8375, in dyld4::LibSystemHelpers::exit(int) const + 11
63: #7    Object "libsystem_c.dylib", at 0x7ff807ce4bfe, in exit + 35
63: #6    Object "libsystem_c.dylib", at 0x7ff807ce4de4, in __cxa_finalize_ranges + 409
63: #5    Object "libdart.6.13.0.dylib", at 0x11b9dd84a, in std::__1::__hash_table<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::__unordered_map_hasher<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::__unordered_map_equal<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> > > >::~__hash_table() + 18
63: #4    Object "libdart.6.13.0.dylib", at 0x11b9dd898, in std::__1::__hash_table<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::__unordered_map_hasher<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::__unordered_map_equal<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> > > >::__deallocate_node(std::__1::__hash_node_base<std::__1::__hash_node<std::__1::__hash_value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<std::__1::shared_ptr<dart::collision::CollisionDetector> ()> >, void*>*>*) + 48
63: #3    Object "libsystem_platform.dylib", at 0x7ff807e01dfc, in _sigtramp + 28
63: #2    Object "libgz-tools2-backward.dylib", at 0x100d21b1b, in backward::SignalHandling::sig_handler(int, __siginfo*, void*) + 13
63: #1    Object "libgz-tools2-backward.dylib", at 0x100d21b89, in backward::SignalHandling::handleSignal(int, __siginfo*, void*) + 79
63: #0    Object "libgz-tools2-backward.dylib", at 0x100d21d17, in backward::StackTraceImpl<backward::system_tag::darwin_tag>::load_from(void*, unsigned long, void*, void*) + 33
63: 
63: [  FAILED  ] CmdLine.RandomSeedValue (2840 ms)
Integration Hydrodynamics test output:
143: [ RUN      ] HydrodynamicsTest.VelocityTestinOil
143: [Wrn] [SystemLoader.cc:73] Trying to load deprecated plugin [ignition-gazebo-physics-system]. Using [gz-sim-physics-system] instead.
143: [Wrn] [Hydrodynamics.cc:345] You are using parameters that may cause instabilities in your simulation. If your simulation crashes we recommend renaming <xUU> -> <xUabsU> and likewise for other axis for more information see:
143: 	[https://github.com/gazebosim/gz-sim/pull/1888](https://github.com/gazebosim/gz-sim/pull/1888%1B[0m%1B[1;33m%1B[0m)
143: [Err] [Conversions.cc:1791] Tried to convert SDF [world] into [plugin]
143: [Wrn] [SystemLoader.cc:73] Trying to load deprecated plugin [ignition-gazebo-physics-system]. Using [gz-sim-physics-system] instead.
143/309 Test #143: INTEGRATION_hydrodynamics ...............................***Exception: SegFault  3.02 sec
test 144

Integration Log System test Output:

243: [ RUN      ] LogSystemTest.LogPaths
243/309 Test #243: INTEGRATION_log_system ..................................***Exception: SegFault  4.07 sec

In the last two weeks these have only happened in mac.monterey machines

Crola1702 avatar Aug 23 '23 15:08 Crola1702

I think these test failures were fixed by merging https://github.com/gazebosim/gz-physics/pull/529 forward to main in https://github.com/gazebosim/gz-physics/pull/532

https://build.osrfoundation.org/job/ignition_gazebo-ci-main-homebrew-amd64/200/ ran successfully on Mac-five.monterey

I think this can be closed

scpeters avatar Aug 28 '23 18:08 scpeters

This issue hasn't happened since 2024-03-22. I'll close it until we see it again

Crola1702 avatar Apr 08 '24 14:04 Crola1702