Wrong error location when "error: StreamTooLong"

[correabuscar]

Zig Version

0.10.0-dev.3844+f7784a081

Steps to Reproduce

1. Create read-input.zig file, in an empty folder, with these contents:

const std = @import("std");

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();

    const stdin = std.io.getStdIn();

    std.debug.print("Enter input: ", .{});
    //std.debug.print("input: ");
    const input = try stdin.reader().readUntilDelimiterAlloc(allocator, '\n', 1);
    defer allocator.free(input);

    std.debug.print("Gotten value: {s}\n", .{input});
}

2. execute $ zig-stage1 run read-input.zig on Gentoo(with this PR already in, otherwise you can't get zig-9999 to emerge), or zig instead of zig-stage1 (the same output results!)
3. type '1' and press Enter at the "Enter input: " prompt

Expected Behavior

$ zig-stage1 run read-input.zig 
Enter input: 1
error: StreamTooLong
/usr/lib/zig/std/io/reader.zig:111:28: 0x20f09a in readUntilDelimiterArrayList (read-input)
            return error.StreamTooLong;
                   ^
/usr/lib/zig/std/io/reader.zig:136:13: 0x20f2a8 in readUntilDelimiterAlloc (read-input)
        try self.readUntilDelimiterArrayList(&array_list, delimiter, max_size);
        ^
./read-input.zig:12:19: 0x20f551 in main (read-input)
    const input = try stdin.reader().readUntilDelimiterAlloc(allocator, '\n', 1);
                  ^

Actual Behavior

$ zig-stage1 run read-input.zig 
Enter input: 1
error: StreamTooLong
/usr/lib/zig/std/io/reader.zig:103:21: 0x20f09a in readUntilDelimiterArrayList (read-input)
            self: Self,
                    ^
/usr/lib/zig/std/io/reader.zig:128:13: 0x20f2a8 in readUntilDelimiterAlloc (read-input)
        pub fn readUntilDelimiterAlloc(
            ^
./read-input.zig:12:19: 0x20f551 in main (read-input)
    const input = try stdin.reader().readUntilDelimiterAlloc(allocator, '\n', 1);
                  ^

[correabuscar]

I get the feeling that this is a more generic bug that once fixed might also fix these two bugs: https://github.com/ziglang/zig/issues/11869 https://github.com/ziglang/zig/issues/12509

[Vexu]

Stack traces get their source locations via a different mechanism so those are unlikely to be related.

[BratishkaErik]

execute $ zig-stage1 run read-input.zig on Gentoo(with this https://github.com/gentoo/gentoo/pull/27112 already in, otherwise you can't get zig-9999 to emerge), or zig instead of zig-stage1 (the same output results!)

I messed up there, zig-stage1 was not stage1 (I fixed this). Here is the output of zig run read-input.zig -fstage1:

Enter input: 1
error: StreamTooLong
/usr/lib/zig/std/io/reader.zig:111:21: 0x235ce8 in std.io.reader.Reader(std.fs.file.File,std.os.ReadError,std.fs.file.File.read).readUntilDelimiterArrayList (read-input)
                    return error.StreamTooLong;
                    ^
/usr/lib/zig/std/io/reader.zig:136:13: 0x2359e2 in std.io.reader.Reader(std.fs.file.File,std.os.ReadError,std.fs.file.File.read).readUntilDelimiterAlloc (read-input)
            try self.readUntilDelimiterArrayList(&array_list, delimiter, max_size);
            ^
/home/bratishkaerik/read-input.zig:12:19: 0x233644 in main (read-input)
    const input = try stdin.reader().readUntilDelimiterAlloc(allocator, '\n', 1);

[correabuscar]

Until now, I had no idea what these stages are all about so I found this info here:

Compiler Stages#

Zig uses multiple compiler stages for bootstrapping the compiler:

    zig0: is just the c++ compiler as a static library
    only implements the backend for build-exe/obj etc
    stage1: is the current compiler, written in C++, compiled with Clang
    uses zig0 library to build pieces of stage2 in (subcommands like translate-c etc)
    stage2: is the current project, written in Zig, compiled with stage1
    stage3: is the fully self-hosted, stage2 code compiled with stage2
    stage1 doesn't implement full optimizations so stage2 binary is not optimized
    stage3 binary is optimized b/c stage2 implements optimizations/much better codegen

I like that Vexu already knew to tag it stage2 even before we realized it was in fact a stage2 output.

As it stands right now:

$ zig run read-input.zig 
Enter input: 1
error: StreamTooLong
/usr/lib/zig/std/io/reader.zig:103:21: 0x20f09a in readUntilDelimiterArrayList (read-input)
            self: Self,
                    ^
/usr/lib/zig/std/io/reader.zig:128:13: 0x20f2a8 in readUntilDelimiterAlloc (read-input)
        pub fn readUntilDelimiterAlloc(
            ^
./read-input.zig:12:19: 0x20f551 in main (read-input)
    const input = try stdin.reader().readUntilDelimiterAlloc(allocator, '\n', 1);
                  ^

$ zig run read-input.zig -fstage1
Enter input: 1
error: StreamTooLong
/usr/lib/zig/std/io/reader.zig:111:21: 0x235ca8 in std.io.reader.Reader(std.fs.file.File,std.os.ReadError,std.fs.file.File.read).readUntilDelimiterArrayList (read-input)
                    return error.StreamTooLong;
                    ^
/usr/lib/zig/std/io/reader.zig:136:13: 0x2359a2 in std.io.reader.Reader(std.fs.file.File,std.os.ReadError,std.fs.file.File.read).readUntilDelimiterAlloc (read-input)
            try self.readUntilDelimiterArrayList(&array_list, delimiter, max_size);
            ^
/home/user/sandbox/zig/zigbyexample/read-input/read-input.zig:12:19: 0x233604 in main (read-input)
    const input = try stdin.reader().readUntilDelimiterAlloc(allocator, '\n', 1);
                  ^
$ zig env
{
 "zig_exe": "/usr/bin/zig",
 "lib_dir": "/usr/lib/zig",
 "std_dir": "/usr/lib/zig/std",
 "global_cache_dir": "/home/user/.cache/zig",
 "version": "0.10.0"
}

[Vexu]

I like that Vexu already knew to tag it stage2 even before we realized it was in fact a stage2 output.

I test each new bug that is reported.

Also here is a reduction:

fn foo(


) type {
    return struct {
        fn bar() !void {
            return error.Bar;
        }
    };
}

test {
    const T = foo();
    try T.bar();
}

Looking at the output of zig ast-check -t a.zig it is clear that the bug is somewhere in AstGen.zig:

         vvvvvvv should be line(6)
[19] bar line(1) hash(184f72f753a5a852768c2421abc758e9): %6 = block_inline({
    %12 = func_inferred([email protected]_type, inferror, body={
    %7 = dbg_block_begin())
    %8 = dbg_stmt(2, 13)
    %9 = dbg_stmt(2, 13)
    %11 = dbg_block_end())
    %10 = ret_err_value("Bar") token_offset:7:26 to :7:29
    }) (lbrace=1:24,rbrace=3:9) node_offset:6:9 to :6:11
    %13 = break_inline(%6, %12)

[correabuscar]

I see that lbrace,rbrace,dbg_stmt are all pointing to the wrong lines(but correct columns), however node_offset points to correct line:column hmmm...

If I do this, it fixes the line numbers for l&rbrace:

Index: /var/tmp/portage/dev-lang/zig-9999/work/zig-9999/src/AstGen.zig
===================================================================
--- .orig/var/tmp/portage/dev-lang/zig-9999/work/zig-9999/src/AstGen.zig
+++ /var/tmp/portage/dev-lang/zig-9999/work/zig-9999/src/AstGen.zig
@@ -3729,7 +3729,7 @@ fn fnDecl(
         defer astgen.fn_block = prev_fn_block;
 
         astgen.advanceSourceCursorToNode(body_node);
-        const lbrace_line = astgen.source_line - decl_gz.decl_line;
+        const lbrace_line = astgen.source_line;
         const lbrace_column = astgen.source_column;
 
         _ = try expr(&fn_gz, params_scope, .none, body_node);
@@ -4179,7 +4179,7 @@ fn testDecl(
     defer astgen.fn_block = prev_fn_block;
 
     astgen.advanceSourceCursorToNode(body_node);
-    const lbrace_line = astgen.source_line - decl_block.decl_line;
+    const lbrace_line = astgen.source_line;
     const lbrace_column = astgen.source_column;
 
     const block_result = try expr(&fn_block, &fn_block.base, .none, body_node);
@@ -10500,7 +10500,7 @@ const GenZir = struct {
             const block = node_datas[fn_decl].rhs;
             const rbrace_start = token_starts[tree.lastToken(block)];
             astgen.advanceSourceCursor(rbrace_start);
-            const rbrace_line = @intCast(u32, astgen.source_line - gz.decl_line);
+            const rbrace_line = @intCast(u32, astgen.source_line);
             const rbrace_column = @intCast(u32, astgen.source_column);
 
             const columns = args.lbrace_column | (rbrace_column << 16);

[topolarity]

I think the print_zir output is a red herring - the numbers shown are actually deltas with respect to the containing decl and print_zir doesn't calculate the absolute line number yet.

The problem seems to be that Sema is generating an anonymous decl with the wrong src_line. Here's a commented version of the example:

// this offset is included
fn foo() type {
    // this offset is *not* included

    const T = struct { // decl.src_line for this struct is 1 (its parent), instead of 4
        // this offset is included
        fn bar() !void {
            // this offset is included
            return error.Bar;
        }
    };
    return T;
}

test {
    const T = foo();
    try T.bar();
}

The ZIR has an AST Node for the struct, but it doesn't have a direct encoding of the src_line delta from the parent. The right fix is probably to update the ZIR encoding for anything that generates an anonymous decl (e.g. struct/union) and update Sema to use this to update the src_line correctly.

[Vexu]

Worth noting that this used to work properly before c76b5c1a31ebe09726cd29e7a6da69613eabfd10 so that fix might have been (at least partially) incorrect.

[topolarity]

Thanks for that pointer!

Looks like before that commit, the src_line is still incorrect for the struct in Sema, but AstGen would calculate deltas w.r.t. the named parent decl so the debug info ended up correct in the end.

That behavior is probably easy enough to restore in AstGen as a workaround, but my intuition is that we should fix up the src_line for anon decls instead (which requires augmenting the ZIR)