Erroneous exit status code reported when using -D

[tlansec]

I have an old script where I'm using subprocess to run YARA on Windows and it reports a bad exit code leading subprocess to think that it broke when it ran successfully:

import subprocess, sys

yara_bin = "yara64" # needs to be on %PATH% or available in current directory
path_to_dotnet_yar = r"/path/to/dotnet_info.yar"

c = subprocess.check_output([yara_bin, path_to_dotnet_yar, "-D", sys.argv[1]])

path_to_dotnet_yar is a path to a YARA rule containing the following rule:

import "dotnet"

rule example_rule
{
condition:
        false
}

When run, this script yields:

subprocess.CalledProcessError: Command '['yara64', 'C:\\Users\\malware\\Documents\\analysis\\dotnet_info.yar', '-D', 'any_dotnet_file_goes_here.bin']' returned non-zero exit status 3221226505.

When this is run normally on the command line, the correct output is produced.

@wxsBSD has pointed me towards this gist (https://gist.github.com/wxsBSD/1e518cef545fee7bb991a9dc6c14a0f7) which suggests I can do the same thing without subprocess, but this probably still should be resolved for the sake of completeness

[tlansec]

Can reproduce same bug on 4.2.0, 4.2.1.

Same bug not reproducable using 4.1.0 binary.

[wxsBSD]

I'm able to reproduce this on 4.2.2, 4.2.1 and 4.2.0-rc1, though 4.2.0-rc1 is interesting, but not on 4.1.3. I think the output of 4.2.0-rc1 is a clue to what is going on:

I'll keep debugging but whatever is going on here is somehow only triggerable via python subprocess, which seems very weird. If I run yara64.exe -D directly I don't see anything wrong.

[wxsBSD]

This can be reproduced without python at all. I'm able to trigger it with yara64 -D ..\always_false.yara ..\737233cb62bcacb17ae4e4ce6f321336b8b8578f26a63da7ea695bc73342e276 | more - redirecting the output to a file or pipeing it to more is important. It seems to go sideways after the first 0x1000 bytes of output:

If I change it to only use import "pe" the same thing happens but not at 0x1000 bytes, so maybe that offset is a red-herring. Here is the output when using only the pe module:

I ran a recursive search of c:\windows and piped that to more and couldn't get anything to go wrong. That narrows it down to just the module data being a problem. I think the problem may lie in https://github.com/VirusTotal/yara/blob/master/cli/yara.c#L1191-L1203 and/or in yr_object_print_data(). Last time we had a similar problem with printing and weird windows behavior it was discussed in https://github.com/VirusTotal/yara/pull/1594 and fixed by @hillu in 1f56e300007fb5f761c89bb2badb52d78df562ac. I'm not saying the two are related, but that there is certainly behavior I can't explain on Windows that involves buffered output.