magma
MME Memmory leak
Prior to submitting an issue, check to see if one has already been created. If there is currently an open issue, add a thumbs-up emoji to identify that it is also affecting you.
Your Environment
- **Version:1.8.0-1663094999-49ed2e69
- **Affected Component: Access Gateway
- **Affected Subcomponent:MME service
- **Deployment Environment:Bare-metal (AGW)
Describe the Issue
I'm testing with Magma and IMS, When my UE try to do a call a message appears on syslog: mme[62101]: LeakSanitizer:DEADLYSIGNAL mme[62101]: ==62101==ERROR: LeakSanitizer: SEGV on unknown address (pc 0x5564acf788fc bp 0x7ff5907e2a80 sp 0x7ff5907e2a50 T15)
To Reproduce
- Feg connected to PCRF.
- Global polices configured.
- UE connected
- Try to make a call.
Expected behavior
If there is some kind of error MME should send an error message, but not to crash by memmory leaks.
Additional context
Jul 5 19:28:54 gw-test-2 sessiond[61936]: I0705 19:28:54.390563 61936 SessionState.cpp:637] Updating used charging credit for Rule=test_apn_internet Rating Group=0 Service Identifier=0 Jul 5 19:28:54 gw-test-2 sessiond[61936]: I0705 19:28:54.390609 61936 SessionState.cpp:637] Updating used charging credit for Rule=test_apn_internet Rating Group=0 Service Identifier=0 Jul 5 19:28:54 gw-test-2 sessiond[61936]: I0705 19:28:54.390841 61936 LocalEnforcer.cpp:433] Received stats for 2 active sessions and 0 stale sessions Jul 5 19:28:55 gw-test-2 smsd[1848]: INFO:root:mme non_eps_service_config is not SMS_ORC8R, sleeping. Jul 5 19:28:55 gw-test-2 mme[62101]: LeakSanitizer:DEADLYSIGNAL Jul 5 19:28:55 gw-test-2 mme[62101]: ==62101==ERROR: LeakSanitizer: SEGV on unknown address (pc 0x5564acf788fc bp 0x7ff5907e2a80 sp 0x7ff5907e2a50 T15) Jul 5 19:28:55 gw-test-2 mme[62101]: ==62101==The signal is caused by a READ memory access. Jul 5 19:28:55 gw-test-2 mme[62101]: ==62101==Hint: this fault was caused by a dereference of a high value address (see register values below). Dissassemble the provided pc to learn which register was used. Jul 5 19:28:56 gw-test-2 mme[62101]: #0 0x5564acf788fc in hashtable_uint64_ts_get_keys /home/vagrant/magma/lte/gateway/c/core/oai/lib/hashtable/hashtable_uint64.c:309 Jul 5 19:28:56 gw-test-2 mme[62101]: #1 0x5564ace5c042 in magma::lte::StateConverter::hashtable_uint64_ts_to_proto(hash_table_uint64_ts_s*, google::protobuf::Map<unsigned long, unsigned long>) /home/vagrant/magma/lte/gateway/c/core/oai/common/state_converter.cpp:217 Jul 5 19:28:56 gw-test-2 mme[62101]: #2 0x5564acb40315 in magma::lte::MmeNasStateConverter::state_to_proto(mme_app_desc_s const, magma::lte::oai::MmeNasState*) /home/vagrant/magma/lte/gateway/c/core/oai/tasks/mme_app/mme_app_state_converter.cpp:707 Jul 5 19:28:56 gw-test-2 mme[62101]: #3 0x5564aca47460 in magma::lte::StateManager<mme_app_desc_s, ue_mm_context_s, magma::lte::oai::MmeNasState, magma::lte::oai::UeContext, magma::lte::MmeNasStateConverter>::write_state_to_db() /home/vagrant/magma/lte/gateway/c/core/oai/include/state_manager.hpp:145 Jul 5 19:28:56 gw-test-2 mme[62101]: #4 0x5564aca2a544 in put_mme_nas_state /home/vagrant/magma/lte/gateway/c/core/oai/tasks/mme_app/mme_app_state.cpp:48 Jul 5 19:28:56 gw-test-2 mme[62101]: #5 0x5564aca0af7b in handle_message /home/vagrant/magma/lte/gateway/c/core/oai/tasks/mme_app/mme_app_main.c:495 Jul 5 19:28:56 gw-test-2 mme[62101]: #6 0x7ff59a6f75c6 in zloop_start (/lib/x86_64-linux-gnu/libczmq.so.4+0x295c6) Jul 5 19:28:56 gw-test-2 mme[62101]: #7 0x5564aca0b7c5 in mme_app_thread /home/vagrant/magma/lte/gateway/c/core/oai/tasks/mme_app/mme_app_main.c:516 Jul 5 19:28:56 gw-test-2 mme[62101]: #8 0x7ff59b1c0608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8608) Jul 5 19:28:56 gw-test-2 mme[62101]: #9 0x7ff59a105132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132) Jul 5 19:28:56 gw-test-2 mme[62101]: LeakSanitizer can not provide additional info. Jul 5 19:28:56 gw-test-2 mme[62101]: SUMMARY: LeakSanitizer: SEGV /home/vagrant/magma/lte/gateway/c/core/oai/lib/hashtable/hashtable_uint64.c:309 in hashtable_uint64_ts_get_keys Jul 5 19:28:56 gw-test-2 mme[62101]: ==62101==ABORTING Jul 5 19:28:56 gw-test-2 systemd[1]: [email protected]: Main process exited, code=exited, status=23/n/a Jul 5 19:28:56 gw-test-2 sessiond[61936]: I0705 19:28:56.388214 61936 SessionState.cpp:582] Reported stat used_tx is less than the current tracked one for IMSI710700000066489-958706, rule_id: test_apn_internet, version: 1 Jul 5 19:28:56 gw-test-2 sessiond[61936]: I0705 19:28:56.389422 61936 SessionState.cpp:637] Updating used charging credit for Rule=test_apn_internet Rating Group=0 Service Identifier=0 Jul 5 19:28:56 gw-test-2 sessiond[61936]: I0705 19:28:56.389602 61936 SessionState.cpp:637] Updating used charging credit for Rule=test_apn_internet Rating Group=0 Service Identifier=0 Jul 5 19:28:56 gw-test-2 sessiond[61936]: I0705 19:28:56.389693 61936 SessionState.cpp:637] Updating used charging credit for Rule=test_apn_internet Rating Group=0 Service Identifier=0 Jul 5 19:28:56 gw-test-2 sessiond[61936]: I0705 19:28:56.389832 61936 LocalEnforcer.cpp:433] Received stats for 2 active sessions and 0 stale sessions Jul 5 19:28:56 gw-test-2 magmad[821]: ERROR:root:GetServiceInfo Error for mme! [StatusCode.UNAVAILABLE] failed to connect to all addresses Jul 5 19:28:56 gw-test-2 systemd[1]: [email protected]: Failed with result 'exit-code'. Jul 5 19:28:58 gw-test-2 sessiond[61936]: I0705 19:28:58.391335 61936 SessionState.cpp:582] Reported stat used_tx is less than the current tracked one for IMSI710700000066489-958706, rule_id: test_apn_internet, version: 1 Jul 5 19:28:58 gw-test-2 sessiond[61936]: I0705 19:28:58.391386 61936 SessionState.cpp:637] Updating used charging credit for Rule=test_apn_internet Rating Group=0 Service Identifier=0 Jul 5 19:28:58 gw-test-2 sessiond[61936]: I0705 19:28:58.391440 61936 SessionState.cpp:637] Updating used charging credit for Rule=test_apn_internet Rating Group=0 Service Identifier=0 Jul 5 19:28:58 gw-test-2 sessiond[61936]: I0705 19:28:58.391487 61936 SessionState.cpp:637] Updating used charging credit for Rule=test_apn_internet Rating Group=0 Service Identifier=0 Jul 5 19:28:58 gw-test-2 sessiond[61936]: I0705 19:28:58.391563 61936 LocalEnforcer.cpp:433] Received stats for 2 active sessions and 0 stale sessions
Full log is attached. AGW_TEST_2_05072023_1416_SYSLOG.log
I am having the same issues. In this case magma is very unstable. How can I solve this?
Normally we are working with older versions of liblsan and gcc-10-base. @javieraubert do you know how to get AGW working with up-to-date versions of liblsan and gcc-10-base packages?