Tofino backend: compiler bug when extracting varbit header

[alexandergall]

Versions:

• open-p4studio: 0f2ad6
• p4c: 660779

Compiling the attached P4 program (as GZIP since GitHub does not allow a .p4 extension) with open-p4studio produces the following error

debian@open-p4studio:~/open-p4studio/foo$ make
[  0%] Built target bf-p4c
[  0%] Built target driver
[100%] Generating bug/tofino/bf-rt.json
running find /home/debian/open-p4studio/foo/bug/tofino \( -name pipe* -or -name graphs -or -name logs -or -name visualization \) -type d -exec rm -rf {} +
running find /home/debian/open-p4studio/foo/bug/tofino \( -path *.json -or -path *.conf -or -path *.p4pp -or -path *.bfa -or -path *.log -or -path *.bin -or -path *.txt \) -type f -exec rm -f {} +
running cc -E -x assembler-with-cpp -D__TARGET_TOFINO__=1 -D__p4c__=1 -D__p4c_major__=9 -D__p4c_minor__=13               -D__p4c_patchlevel__=4 -D__p4c_version__=\"9.13.4\" -C -undef -nostdinc -x assembler-with-cpp -I /home/debian/open-p4studio/install/share/p4c/p4include -o /home/debian/open-p4studio/foo/bug/tofino/bug.p4pp /home/debian/open-p4studio/bug.p4
running /home/debian/open-p4studio/install/bin/p4c-barefoot --nocpp -D__TARGET_TOFINO__=1 -D__p4c__=1 -D__p4c_major__=9 -D__p4c_minor__=13               -D__p4c_patchlevel__=4 -D__p4c_version__=\"9.13.4\" -g -I /home/debian/open-p4studio/install/share/p4c/p4include --p4v=16 --target tofino --arch tna -o /home/debian/open-p4studio/foo/bug/tofino /home/debian/open-p4studio/foo/bug/tofino/bug.p4pp --create-graphs --verbose -Ttable_placement:3,table_summary:1,table_dependency_graph:3,table_dependency_summary:3,allocate_phv:1,allocator_base:3,trivial_allocator:5,greedy_allocator:5,alias:1,,allocate_clot:1,clot_info:1,split_parser_state:1,allocate_parser_match_register:1,allocate_parser_checksum:1,lower_parser:1,decaf*:1,characterize_parser.h:1,bridged_packing:1,ixbar_info:3 -Tstage_advance:3>/home/debian/open-p4studio/foo/bug/tofino/stage_adv.log --bf-rt-schema bug/tofino/bf-rt.json

1 error, 0 warnings generated.

In file: /home/debian/open-p4studio/pkgsrc/p4-compilers/p4c/ir/ir-inline.h:90
Compiler Bug: /home/debian/open-p4studio/pkgsrc/p4-compilers/p4c/ir/ir-inline.h:90: Null n

Internal compiler error. Please submit a bug report with your code.
make[2]: *** [CMakeFiles/bug-tofino.dir/build.make:80: bug/tofino/bf-rt.json] Error 4
make[1]: *** [CMakeFiles/Makefile2:127: CMakeFiles/bug-tofino.dir/all] Error 2
make: *** [Makefile:149: all] Error 2

The error only occurs when extracting a varbit header field.

The build was prepared with

debian@open-p4studio:~/open-p4studio/foo$ cmake ../p4studio/ -DCMAKE_INSTALL_PREFIX=$SDE_INSTALL -DCMAKE_MODULE_PATH=$SDE/cmake -DP4FLAGS="-v" -DP4_NAME=bug -DP4_PATH=/home/debian/open-p4studio/bug.p4
--
Using SDE: /home/debian/open-p4studio
--
Using SDE_INSTALL: /home/debian/open-p4studio/install
-- Python3 found: /usr/bin/python3.9
-- Python3 version: 3.9.2
-- Python3 include dir: /usr/include/python3.9
-- Python3 libraries: /usr/lib/x86_64-linux-gnu/libpython3.9.so
--
P4_LANG: p4-16
P4C: /home/debian/open-p4studio/install/bin/bf-p4c
P4C-GEN_BRFT-CONF: /home/debian/open-p4studio/install/bin/p4c-gen-bfrt-conf
P4C-MANIFEST-CONFIG: /home/debian/open-p4studio/install/bin/p4c-manifest-config
--
P4_PATH: /home/debian/open-p4studio/bug.p4
P4_NAME: bug
--
P4FLAGS: -v
-- Configuring done
-- Generating done
-- Build files have been written to: /home/debian/open-p4studio/foo

bug.p4.gz

[vgurevich]

@alexandergall -- just to make sure: would you expect the compiler to mark the code in the state foo:

    state foo {
        pkt.extract(hdr.foo, (bit<32>)1);
        transition accept;
    }

as invalid and return an error (because only whole bytes can be extracted) or something else?

[alexandergall]

The expected result is

error: Varbit field size expression evaluates to non byte-aligned value 1: pkt.extract(hdr.foo, 32w1)

This is what p4c from SDE 9.13.4 produces, which doesn't exhibit this bug.

[jafingerhut]

The expected result is

error: Varbit field size expression evaluates to non byte-aligned value 1: pkt.extract(hdr.foo, 32w1)

This is what p4c from SDE 9.13.4 produces, which doesn't exhibit this bug.

Is there a fix for this in SDE 9.13.4 source code that is not in the public p4lang/p4c repository that might address this, that someone at Intel could contribute?

Or is this issue perhaps introduced by changes in the p4lang/p4c Tofino backend code since it was released to the public?

[vgurevich]

I checked and both SDE-9.13.3 and even 9.13.1 produce the expected error message for the provided program.

Therefore, this looks like an issue that was inadvertently introduced in the public backend.

[fruffy]

Getting the stacktrace of this particular exception could be helpful. I currently do not have the bf-p4c binary at hand.

[alexandergall]

How do I enable stack traces for exceptions?

[vlstill]

How do I enable stack traces for exceptions?

@alexandergall maybe -Tcrash:1 would work also for the Tofino compiler.

[fruffy]

How do I enable stack traces for exceptions?

If Vladimir's suggestion does not work you could install https://github.com/ianlancetaylor/libbacktrace and recompile with it enabled. Or use gdb with catch exception and backtrace.

[vgurevich]

I use GDB with catch throw and bt, same as @fruffy suggested.

However, by default the compiler is built without debug symbols, so you will need to rebuild it accordingly, e.g. with -DCMAKE_BUILD_TYPE=RelWithDebInfo

Here is the output I have for @alexandergall 's program:

(gdb) catch throw
Catchpoint 1 (throw)
(gdb) run --target tofino bug.p4
Starting program: /home/ubuntu/open-p4studio/install/bin/p4c-barefoot --target tofino bug.p4
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[Detaching after vfork from child process 146683]

Catchpoint 1 (exception thrown), 0x00007ffff7430662 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0  0x00007ffff7430662 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x000055555e724779 in P4::IR::Vector<P4::IR::Expression>::visit_children (this=0x7ffff6b4e650, v=...)
    at /usr/include/c++/9/bits/stl_iterator.h:803
#2  0x000055555e48b5ff in std::__do_visit<> (__visitor=...) at /usr/include/c++/9/variant:1638
#3  std::visit<P4::IR::Annotation::visit_children(P4::Visitor&)::<lambda(auto:25&&)>, std::variant<P4::IR::Vector<P4::IR::AnnotationToken>, P4::IR::Vector<P4::IR::Expression>, P4::IR::IndexedVector<P4::IR::NamedExpression> >&> (__visitor=...)
    at /usr/include/c++/9/variant:1663
#4  P4::IR::Annotation::visit_children (this=<optimized out>, v=...)
    at /home/ubuntu/open-p4studio/build/pkgsrc/p4-compilers/p4c/ir/ir-generated.cpp:821
#5  0x000055555ecf2f9d in P4::Transform::apply_visitor (this=0x7fffffffcf50, n=0x7ffff6b4e730, name=<optimized out>)
    at /home/ubuntu/open-p4studio/pkgsrc/p4-compilers/p4c/ir/visitor.cpp:557
#6  0x000055555e726d60 in P4::IR::Vector<P4::IR::Annotation>::visit_children (this=this@entry=0x7ffff6b42698, v=...)
    at /usr/include/c++/9/bits/stl_iterator.h:803
#7  0x000055555e5361fc in P4::IR::ParserState::visit_children (this=0x7ffff6b42600, v=...)
    at /home/ubuntu/open-p4studio/build/pkgsrc/p4-compilers/p4c/ir/ir-generated.cpp:4985
#8  0x000055555ecf3419 in P4::Transform::apply_visitor (this=0x7fffffffcf50, n=0x7ffff6c70000, name=<optimized out>)
    at /home/ubuntu/open-p4studio/pkgsrc/p4-compilers/p4c/ir/visitor.cpp:584
#9  0x000055555e78b9cf in P4::IR::IndexedVector<P4::IR::ParserState>::visit_children (this=0x7ffff6ba4158, v=...)
    at /usr/include/c++/9/bits/stl_iterator.h:803
#10 0x000055555ecf3419 in P4::Transform::apply_visitor (this=0x7fffffffcf50, n=0x7ffff6b3c7e0, name=<optimized out>)
    at /home/ubuntu/open-p4studio/pkgsrc/p4-compilers/p4c/ir/visitor.cpp:584
#11 0x000055555e721110 in P4::IR::Vector<P4::IR::Node>::visit_children (this=0x7ffff6c61c80, v=...)
    at /usr/include/c++/9/bits/stl_iterator.h:803
#12 0x000055555ecf3419 in P4::Transform::apply_visitor (this=0x7fffffffcf50, n=0x7ffff6c61d20, name=<optimized out>)
    at /home/ubuntu/open-p4studio/pkgsrc/p4-compilers/p4c/ir/visitor.cpp:584
#13 0x000055555ecfef38 in P4::Visitor::visit (this=this@entry=0x7fffffffcf70, n=@0x7fffffffcea8: 0x7ffff6c61d20, 
    name=name@entry=0x0) at /home/ubuntu/open-p4studio/pkgsrc/p4-compilers/p4c/ir/visitor.cpp:773
#14 0x000055555e48dd97 in P4::IR::P4Program::apply (this=this@entry=0x7ffff6c61d20, v=..., ctxt=ctxt@entry=0x0)
    at /home/ubuntu/open-p4studio/build/pkgsrc/p4-compilers/p4c/ir/ir-generated.cpp:6209
#15 0x000055555d48998b in P4::IR::P4Program::apply (ctxt=0x0, v=..., this=0x7ffff6c61d20)
    at /home/ubuntu/open-p4studio/build/pkgsrc/p4-compilers/p4c/ir/ir-generated.h:8218
#16 run_frontend () at /home/ubuntu/open-p4studio/pkgsrc/p4-compilers/p4c/backends/tofino/bf-p4c/frontend.cpp:121
#17 0x000055555d412230 in main (ac=<optimized out>, av=<optimized out>)
    at /home/ubuntu/open-p4studio/pkgsrc/p4-compilers/p4c/backends/tofino/bf-p4c/p4c-barefoot.cpp:444
(gdb) c
Continuing.

1 error, 0 warnings generated.

In file: /home/ubuntu/open-p4studio/pkgsrc/p4-compilers/p4c/ir/ir-inline.h:90
Compiler Bug: /home/ubuntu/open-p4studio/pkgsrc/p4-compilers/p4c/ir/ir-inline.h:90: Null n

Internal compiler error. Please submit a bug report with your code.
[Inferior 1 (process 146679) exited with code 04]

[alexandergall]

Thanks for the hints. I can reproduce @vgurevich's result.

[fruffy]

This looks like a bug introduced by #5018. We are probably accessing an annotation in an unsafe way.

[asl]

This looks like a bug introduced by #5018. We are probably accessing an annotation in an unsafe way.

It will be another exception then, thrown in std::visit. Here the BUG_CHECK is raised from Vector – one of children is null.

[fruffy]

Fixed in #5193 but requires some more testing.