segfault/memory corruption when piping buffer contents through a command in FILE_SAVE_PRE hook

[zenhack]

Given this visrc.lua:

require("vis")

vis.events.subscribe(vis.events.FILE_SAVE_PRE, function(file, path)
   vis:command(",|cat")
end)

I get frequent memory-corruption related errors after saving a file; sometimes double-free assertions, sometimes segfaults. I originally hit this trying to use goimports, but per the above it triggers even when used with a command that doesn't actually change the text. I can reproduce this on both v0.4 and master.

[martanne]

Thanks for the report, the issue is that the transcript structure only exists once per file. As a result it gets corrupted in these nested commands. The write command will initialize (and free it) while in the meantime the cat command does so too.

This will require some code restructuring to fix (i.e. there should be one transcript per file and command).

Furthermore, there are some semantic issue to deal with. Currently the address of the write command (by default the whole file) is calculated before the hook is executed. It will thus only write the range corresponding to the initial file size. In case the hook adds additional text it is silently ignored. If the hook removes text, the write fails because not enough data was available.

We could try to delay the address calculation until after the hook is completed. In any case this needs a bit more thoughts.

[bugabinga]

This can be reliably reproduced on OpenBSD 6.3+. Attaching core dump: core.zip

[hyphenrf]

don't know if related but this also causes a similar segfault:

vis:command_register('paste',
    function() vis:command('<cat /dev/tty') end,
    'paste mode, end with C-d'
)

[ghost]

@hyphenrf I believe it deserves a separate issue. Also related - #357