Python API does not work for emitting values created with the python API

[gsa-tspaere]

Hey!

First of all, thanks for the awesome library! For parsing it's working great, but I'm having some real issues with creating yaml trees for emitting. I am currently running into the following issue:

import ryml
import time
import datetime as dt


def wow(othertree):
    child_id = othertree.append_child(othertree.root_id())

    datetime = dt.datetime.now()
    string_version = datetime.isoformat()
    print("Adding: ", string_version)

    othertree.to_val(child_id, string_version, ryml.VAL_PLAIN)

# Because just using ryml.Tree() results in segfault
othertree = ryml.parse_in_arena("")
othertree.to_seq(othertree.root_id())
for i in range(3):
    wow(othertree)
    time.sleep(0.1)

output = ryml.emit_yaml_malloc(othertree, othertree.root_id())
print(output)

The output from running this is:

Adding:  2025-11-25T11:08:54.862304
Adding:  2025-11-25T11:08:54.962477
Adding:  2025-11-25T11:08:55.062681
- 2025-11-25T11:08:55.062681
- 2025-11-25T11:08:55.062681
- 2025-11-25T11:08:55.062681

What is the proper way to do this? It's worth noteting that when done inline, the output is not reproducable:

import ryml
import time
import datetime as dt

othertree = ryml.parse_in_arena("")
othertree.to_seq(othertree.root_id())

for i in range(3):
    child_id = othertree.append_child(othertree.root_id())

    datetime = dt.datetime.now()
    string_version = datetime.isoformat()
    print("Adding: ", string_version)

    othertree.to_val(child_id, string_version, ryml.VAL_PLAIN)

    time.sleep(0.1)

output = ryml.emit_yaml_malloc(othertree, othertree.root_id())
print(output)

With this the output is:

Adding:  2025-11-25T11:11:20.634962
Adding:  2025-11-25T11:11:20.735134
Adding:  2025-11-25T11:11:20.835368
- 2025-11-25T11:11:20.835368
- 

[biojppm]

You are adding temporaries to the tree. Thereafter, python is free to delete or overwrite those temporaries, but the tree will still be referencing their (now stale) memory! The first example works only by accident.

Do this for example:

    # keep the scalars at same scope as the tree.
    # don't modify or delete while using the tree!
    scalars = [dt.datetime.now().isoformat() for _ in range(3)]
    tree = ryml.parse_in_arena("")
    root = tree.root_id()
    tree.to_seq(root)
    for s in scalars:
        child_id = tree.append_child(root)
        tree.to_val(child_id, s, ryml.VAL_PLAIN)
    output = ryml.emit_yaml_malloc(tree, root)
    print(output)

Always bear in mind that the rapidyaml tree does not take ownership of any of the scalars which you give it. You will have to keep the scalars around for as long as the tree is there.

This is not ergonomic in python, but is a consequence of the design choices in C++ to achieve the speed. In the C++ version there is tree.to_arena() to make life easier in these situations, but that is not yet implemented in python. (That function serializes values to the tree's arena, enabling the caller to use temporaries at the cost of a serialization/string copy.)

[biojppm]

As for the segfault, I was able to reproduce it. Thanks for reporting that!

[gsa-tspaere]

Hey, thanks for the answer!

That makes sense, is there currently a plan to incorporate the to_arena functionality on the python side, or is this a more long term feature?

Once again, thanks for the answer!

[biojppm]

Indeed, I am working on that in #560 , which should be coming soon.