Segmentation fault in PATH Codec encode

[111pontes]

Script

#!/usr/bin/env python3

from ydk.path import Repository, Capability, Codec
from ydk.types import EncodingFormat

payload = '''
{
  "openconfig-interfaces:interfaces": {
    "interface": [
      {
        "name": "Loopback0",
        "config": {
          "name": "Loopback0",
          "description": "Lo0 interface description",
          "mtu": 1500
        }
      }
    ]
  }
}'''

repo = Repository('/home/host/.ydk/router')
caps = [Capability('openconfig-interfaces', '')]
root_schema = repo.create_root_schema(caps)

codec = Codec()

data_node = codec.decode(root_schema, payload, EncodingFormat.JSON)

print(codec.encode(data_node, EncodingFormat.JSON, True))

Output:

user@host$ ./test.py 
{
  "openconfig-interfaces:interfaces": {
    "interface": [
      {
        "name": "Loopback0",
        "config": {
          "name": "Loopback0",
          "description": "Lo0 interface description",
          "mtu": 1500
        }
      }
    ]
  }
}

Segmentation fault
user@host$ 

System:

Ubuntu Bionic running:

user@host$ pip list | grep "ydk "
ydk                      0.8.4
user@host$ 

[111pontes]

If code block is executed under Ptyhon main block:

if __name__ == "__main__":

No segfault is produced, but execution hangs.

[ygorelik]

Adding stack trace:

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_pthread.dylib       	0x00007fff6c343137 pthread_mutex_lock + 0
1   ydk_.so                       	0x0000000109378782 lydict_remove + 82 (dict.c:127)
2   ydk_.so                       	0x0000000109416890 lyd_free + 496
3   ydk_.so                       	0x0000000109416724 lyd_free + 132 (tree_data.c:4984)
4   ydk_.so                       	0x0000000109416724 lyd_free + 132 (tree_data.c:4984)
5   ydk_.so                       	0x000000010941b8d4 lyd_free_withsiblings + 164 (tree_data.c:5047)
6   ydk_.so                       	0x000000010928db2a ydk::path::DataNodeImpl::~DataNodeImpl() + 58 (data_node.cpp:82)
7   ydk_.so                       	0x00000001092e829d ydk::path::RootDataImpl::~RootDataImpl() + 77 (root_data_node.cpp:44)
8   ydk_.so                       	0x00000001092e82c5 ydk::path::RootDataImpl::~RootDataImpl() + 21 (root_data_node.cpp:44)
9   ydk_.so                       	0x00000001092c42a9 std::__1::__shared_ptr_emplace<ydk::path::RootDataImpl, std::__1::allocator<ydk::path::RootDataImpl> >::__on_zero_shared() + 41 (memory:3707)
10  ydk_.so                       	0x0000000109059dda pybind11::class_<ydk::path::DataNode, std::__1::shared_ptr<ydk::path::DataNode> >::dealloc(pybind11::detail::value_and_holder&) + 90
11  ydk_.so                       	0x0000000109025cfb pybind11::detail::clear_instance(_object*) + 411
12  ydk_.so                       	0x000000010901f0bf pybind11_object_dealloc + 15
13  org.python.python             	0x00000001089e7da2 free_keys_object + 127
14  org.python.python             	0x00000001089ebb8d dict_clear + 9

The stack shows that error appears in Libyang code during release of memory for root data node.

Note. This issue was not reproduced in C++ code with the same data. The Valgrind also does not show any memory leaks while running the test.

[ygorelik]

The root cause At the end of the python process, when all the objects are getting destroyed, the root schema node object root_schema got destroyed before the data_node. That is unacceptable. All the data node objects must be destroyed before the root schema node object. That is taking care of when root schema is created under NetconfSession object. The C++ program also takes care of the sequence of destroying allocated objects.

Solution In order to assure correct sequence of destroying objects all the operations on the data nodes must be performed inside some function. Here is an example, how the issue can be resolved:

def perform_operation(root, data):
    codec = Codec()
    data_node = codec.decode(root, data, EncodingFormat.JSON)
    print(codec.encode(data_node, EncodingFormat.JSON, True))

if __name__ == '__main__':
    repo = Repository('/home/host/.ydk/router')
    caps = [Capability('openconfig-interfaces', '')]
    root_schema = repo.create_root_schema(caps)
    perform_operation(root_schema, payload)

With this workaround the issue can be closed as there is nothing to fix in the YDK code.

[111pontes]

Thanks for the explanation. This situation should be handled automatically for the user. Let's keep the issue open. Thanks again!