Runtime segfault occurs when using Iters

[MightyAlex200]

I was trying to implement an algorithm in Pony when I encountered a segfault in my program while trying to use Iters. The reason for the segfault is invalid address (fault address: 0xffffffff), according to lldb.

Here is the code in a Gist. (note: hashgraph.pony and event.pony are located in a directory hashgraph/, but Gist does not support directories)

The segfault begins in the Hashgraph.famous function (called in main.pony, line 11), when nth is called on the Iter (hashgraph.pony, line 138), and the last of my code that is run before the segfault appears to be line 55 of hashgraph.pony (a fold call on the Iter). The last call before the segfault appears to be in iter.pony, in the filter_stateful function, line 121.

I have no idea how to recreate this issue in simpler code, nothing in my code seems particularly dangerous. I am using ponyc 0.23.3 [release], but have not been able to try this with the master branch (though iter.pony has not been commited to since 2017).

[mfelsche]

Can you provide a full lldb backtrace? This way it is easier to spot the problem.

[MightyAlex200]

Here is the backtrace of the executable compiled with the docker image:

(lldb) thread backtrace 2
* thread #2, name = 'main', stop reason = signal SIGSEGV: invalid address (fault address: 0xffffffff)
  * frame #0: 0x00000000ffffffff
    frame #1: 0x000000000040dd11 main`itertools_$6$47_hashgraph_Event_String_val_I64_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_val_ref__find_next_o(this=0x00007fffe59d1b80) at iter.pony:121
    frame #2: 0x000000000040de9f main`itertools_$6$47_hashgraph_Event_String_val_I64_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_val_ref_has_next_b(this=0x00007fffe59d1b80) at iter.pony:134
    frame #3: 0x000000000040e0f9 main`itertools_Iter_hashgraph_Event_String_val_I64_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_val_ref_has_next_b(this=0x00007fffe59d1ba0) at iter.pony:13
    frame #4: 0x00000000004121c9 main`itertools_$6$44_hashgraph_Event_String_val_I64_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_ref_has_next_b(this=0x00007fffe59d0c40) at iter.pony:100
    frame #5: 0x0000000000403ae7 main`itertools_Iter_u4_Alice_val_Bob_val_Carol_val_Dave_val_ref_fold_collections_HashSet_u4_Alice_val_Bob_val_Carol_val_Dave_val_collections_HashEq_u4_Alice_val_Bob_val_Carol_val_Dave_val_val_ref_ooo(this=0x00007fffe59d0c60, acc=0x00007fffe59d1bc0, f=0x0000000000428010) at iter.pony:491
    frame #6: 0x000000000040c44d main`hashgraph_Hashgraph_String_val_I64_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_ref_calculate_population_oZ(this=0x00007ffff71db3c0, events'=0x00007fffe59d1ba0) at hashgraph.pony:55
    frame #7: 0x000000000040d588 main`hashgraph_Hashgraph_String_val_I64_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_ref_many_creators_ob(this=0x00007ffff71db3c0, events'=0x00007fffe59d1ba0) athashgraph.pony:61
    frame #8: 0x000000000040e5fe main`hashgraph_$2$16_String_val_I64_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_ref_apply_ob(this=0x00007fffe59d2180, y=0x00007fffe59db980) at hashgraph.pony:110
    frame #9: 0x000000000040dd11 main`itertools_$6$47_hashgraph_Event_String_val_I64_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_val_ref__find_next_o(this=0x00007fffe59d21e0) at iter.pony:121
    frame #10: 0x000000000040de9f main`itertools_$6$47_hashgraph_Event_String_val_I64_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_val_ref_has_next_b(this=0x00007fffe59d21e0) at iter.pony:134
    frame #11: 0x000000000040f48f main`itertools_$6$51_hashgraph_Event_String_val_I64_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_val_Bool_val_ref_has_next_b(this=0x00007fffe59d2240) at iter.pony:181
    frame #12: 0x000000000040f8b5 main`itertools_Iter_Bool_val_ref_nth_Zb(this=0x00007fffe59d2260, n=1) at iter.pony:593
    frame #13: 0x000000000040d7fa main`hashgraph_Hashgraph_String_val_I64_val_u4_Alice_val_Bob_val_Carol_val_Dave_val_ref_famous_oo(this=0x00007ffff71db3c0, x=0x00007fffe59db880) at hashgraph.pony:138
    frame #14: 0x00000000004039c9 main`Main_tag_create_oo(this=0x00007ffff71db800, env=0x00007ffff71db000) at main.pony:11
    frame #15: 0x00000000004024fa main`Main_Dispatch + 122
    frame #16: 0x0000000000416142 main`ponyint_actor_run + 402
    frame #17: 0x000000000041c8f1 main`run_thread + 209
    frame #18: 0x00007ffff7bc108c libpthread.so.0`start_thread + 220
    frame #19: 0x00007ffff74dde7f libc.so.6`__GI___clone + 63

[MightyAlex200]

Just compiled this with the master branch, segfault still occurs.

[jemc]

Thanks for reporting!

This looks like a nasty one, so I've added the "triggers release" label.

[MightyAlex200]

I've created a generalized version that also segfaults:

use "itertools"

class SegFaulter[A]
  fun ref segfault() =>
    Iter[U8]([as U8: 1; 2; 3].values())
      .map[None]({(a) =>
        Iter[SegFaulter[A]]([SegFaulter[A]].values())
          .map[None]({(b) => None }).run()
      })
      .run()

actor Main
  new create(env: Env) =>
    SegFaulter[None].segfault()

It won't work without the generic, so I think it's safe to say they're involved.

However, this occasionally compiles to a version with no segfault even though no changes to the code were made.

I have attached 2 Linux 64-bit binaries that were compiled the exact same way (ponyc --debug) with the same code. One of them segfaults and the other does not. segfault.zip

[jemc]

Interestingly enough, I think I've found a type system bug that was hiding a capability safety issue in filter_stateful. I'll try to figure out if it is related.

[SeanTAllen]

The type system bug that @jemc mentioned was fixed several years ago but this is still segfaulting and as such, still needs investigation.