stage1: miscompilation on writing with comptime-computed offsets into comptime constructed buffer

[matu3ba]

Zig Version

0.10.0-dev.787+5aa35f62c

Steps to Reproduce

Run

zig build-exe comptimeMiscompilation.zig
./comptimeMiscompilation

Inspect with an editor the binary comptimeMiscompilation

File comptimeMiscompilation.zig:

const std = @import("std");
const mem = std.mem;

const strings = [_][]const u8{ "Hello", "There" };
pub fn main() !void {
    comptime var buf_size = 0;
    inline for (strings) |str| {
        buf_size += str.len;
    }
    const len_offsets = strings.len;
    comptime var buf = [_]u8{0} ** buf_size;
    comptime var offsets = [_]u64{0} ** len_offsets;
    //@compileLog(strings.len);
    //@compileLog(strings[0].len);
    //@compileLog(strings[1].len);
    //@compileLog(buf);
    //@compileLog(offsets);
    comptime var cur_offset = 0;
    inline for (strings) |str, i| {
        //@compileLog(cur_offset);
        offsets[i] = cur_offset;
        //@compileLog(cur_offset);
        //@compileLog(str.len);
        //@compileLog(cur_offset + str.len);
        //@compileLog(str.len);
        //@compileLog(buf[cur_offset .. cur_offset + str.len].len);
        mem.copy(u8, buf[cur_offset .. cur_offset + str.len], str);
        cur_offset += str.len;
    }
    std.debug.print("{d}\n", .{buf_size});
    std.debug.print("{d}\n", .{len_offsets});
    std.debug.print("{s}\n", .{buf});
    std.debug.print("{d}\n", .{offsets});
    std.debug.print("{d}\n", .{cur_offset});
}

Expected Behavior

1. No error. I checked the offsets with the debug logs and they are correct. Without mem.copy the program works.
2. Even if not, the program should crash at comptime.
3. If comptime fails (ie due to versatility of stage1), it should crash instead of writing the crash data into the binary that is being linked.

Actual Behavior

Output of the broken comptime execution is written into the resulting elf binary (on Linux) stripped by the nonvisual parts:

{ ... }ELF/proc/self/exe.debug_info.debug_abbrev.debug_str.debug_line.debug_rangessentinel mismatch
attempt to cast negative value to unsigned integerPanicked during a panic. Aborting.
Unable to dump stack trace: Unable to open debug info: {s}
ZIG_DEBUG_COLORNO_COLORTERMdumbexact division produced remainderUnable to dump stack trace: {s}
shift amount is greater than the type size{s}:{d}:{d}0x{x} in {s} ({s})unexpected errno: {d}
Segmentation fault at address 0x{x}
Illegal instruction at address 0x{x}
Bus error at address 0x{x}
{d}
{s}
Unable to dump stack trace: Unable to open debug info: {s}

Output is

Segmentation fault at address 0x202cd8
/home/user/dev/git/zig/zig/master/lib/std/mem.zig:221:19: 0x212508 in std.mem.copy (comptim
eMiscompilation)
        dest[i] = s;
                  ^
/home/user/dev/git/zig/tryzig/comptimeMiscompilation.zig:27:17: 0x22d370 in main (comptimeM
iscompilation)
        mem.copy(u8, buf[cur_offset .. cur_offset + str.len], str);
                ^
/home/user/dev/git/zig/zig/master/lib/std/start.zig:561:37: 0x2268aa in std.start.callMain
(comptimeMiscompilation)
            const result = root.main() catch |err| {
                                    ^
/home/user/dev/git/zig/zig/master/lib/std/start.zig:495:12: 0x20745e in std.start.callMainW
ithArgs (comptimeMiscompilation)
    return @call(.{ .modifier = .always_inline }, callMain, .{});
           ^
/home/user/dev/git/zig/zig/master/lib/std/start.zig:409:17: 0x2064f6 in std.start.posixCall
MainAndExit (comptimeMiscompilation)
    std.os.exit(@call(.{ .modifier = .always_inline }, callMainWithArgs, .{ argc, argv, envp }));
                ^
/home/user/dev/git/zig/zig/master/lib/std/start.zig:322:5: 0x206302 in std.start._start (co
mptimeMiscompilation)
    @call(.{ .modifier = .never_inline }, posixCallMainAndExit, .{});
    ^
Abgebrochen (Speicherabzug geschrieben)

Most likely the failures are caused by one or multiple offset-by-1 error. Speculatively there is missing/errorneous handling of the result and llvm doing bad stuff, but debugging is needed. (And unfortunately I dont have time for this in the next month.)

[squeek502]

Potentially related to https://github.com/ziglang/zig/issues/10684?

[matu3ba]

I tried to debug this, but I got 4 or 5 panics during panic and in sight of stage2 getting functional, I did not try to fix this.

[mlugg]

There's actually a bug in your program here. comptime vars are not mutable at runtime, and are put into .rodata after semantic analysis. This test case tries to perform the mem.copy - mutating buf- at runtime. The reason that the compiler doesn't catch it is a little subtle. Consider this code:

test {
    comptime var x: u32 = undefined;
    const p = &x;
    @compileLog(@TypeOf(p));
    p.* = runtimeVal();
}

The assignment of p.* here (correctly) gives a compilation error, since Sema expands the assignment at comptime (because p is comptime-known) and realises that p.* can only be assigned at comptime. However, what does the @compileLog print? Well, even if it's only usable at comptime, p is still a perfectly valid mutable pointer: so it has type *u32. Here's the problem: what happens if we pass p into a runtime function?

const std = @import("std");

test {
    comptime var x: u32 = undefined;
    const p = &x;
    runtimeSet(p);
    std.log.info("{}", .{x});
}

fn runtimeSet(p: *u32) void {
    p.* = 42;
}

Disaster strikes! Because p isn't comptime-known within the body of runtimeSet, it never realises that it's only comptime-mutable, Type checking succeeds, the binary compiles, but it segfaults at runtime because p points to read-only data. I'm not 100% sure what the correct solution looks like here. I think it would probably be to demote runtime-immutable pointers to const pointers when passed through to runtime code; that means when they're passed as a non-comptime parameter to a non-inline function, or when being assigned to a global.

EDIT: even simpler trigger! We can make the pointer runtime-known simply by putting it in a runtime var.

test {
    comptime var x: u32 = 0;
    var p = &x; // p is now runtime-known!
    p.* = runtimeVal(); // segfault
}

fn runtimeVal() u32 {
    return 42;
}

[mlugg]

The language footgun here was solved by #19414, and the underlying issue has decent test coverage.