medium memory model doesn't seem to work

[stsp]

ia16.tar.gz

Hi, I wanted to see how the medium model works. Attached is a trivial test-case that just throws dozens of errors. Am I doing something wrong?

[asiekierka]

What are the errors? What build of the build-ia16 toolchain are you using (self-built, from PPA, etc.)?

[stsp]

I am using PPA. The errors are (but you can just run the test-case, its trivial):

/usr/lib/x86_64-linux-gnu/gcc/ia16-elf/6.3.0/../../../../../ia16-elf/bin/ld: a.out section `.data' will not fit in region `dsegvma'
/usr/lib/x86_64-linux-gnu/gcc/ia16-elf/6.3.0/../../../../../ia16-elf/bin/ld: region `dsegvma' overflowed by 39264 bytes
/usr/lib/x86_64-linux-gnu/gcc/ia16-elf/6.3.0/../../../../../ia16-elf/lib/medium/libc.a(lib_a-impure.o):(.rodata+0x0): relocation truncated to fit: R_386_16 against `.data'
/usr/lib/x86_64-linux-gnu/gcc/ia16-elf/6.3.0/../../../../../ia16-elf/lib/medium/libc.a(lib_a-environ.o):(.data+0x0): relocation truncated to fit: R_386_16 against `.bss'

... and more and more of the alike.

[stsp]

Ah by copying the messages, I've noticed that I am overflowing .data, not .text... Let me see if I can "fix" that.

[asiekierka]

The C compiler emits special segmentation information in the medium memory model mode (in different formats, depending on if you use the segelf mode or not), so using objcopy to inject a payload is probably insufficient - and - as this is not the huge memory model - single data structures larger than 64K won't be supported. For my own purposes, I use a bin2c-style tool to convert binary files to .c/.h pairs.

[stsp]

ia16.tar.gz You are right, I was testing huge (or large) memory model, which was wrong. I fixed the test-case to only overflow the .text section, attached.

Now it gives this:

ia16-elf-gcc -mcmodel=medium main.o payload.o
/usr/lib/x86_64-linux-gnu/gcc/ia16-elf/6.3.0/../../../../../ia16-elf/bin/ld: a.out section `.text' will not fit in region `csegvma'
/usr/lib/x86_64-linux-gnu/gcc/ia16-elf/6.3.0/../../../../../ia16-elf/bin/ld: Error: too large for a small-model .exe file.
/usr/lib/x86_64-linux-gnu/gcc/ia16-elf/6.3.0/../../../../../ia16-elf/bin/ld: region `csegvma' overflowed by 37248 bytes
main.o: in function `main':
(.text+0x8): relocation truncated to fit: R_386_16 against `_binary_payload_tmp_size'
(.text+0xc): relocation truncated to fit: R_386_16 against `_binary_payload_tmp_end'

Please note this part: Error: too large for a small-model .exe file. It is still trying to create "small-model .exe file".

[andrewbird]

Don't you need to tell the Makefile how to compile main.c into main.o, else it will be just using compiler defaults?

[stsp]

I tried -mcmodel=medium when compiling main.o but this didn't change anything. Just to be certain, attached is such test-case. ia16.tar.gz

ia16-elf-gcc -mcmodel=medium   -c -o main.o main.c
dd if=/dev/random of=payload.tmp bs=1024 count=100
ia16-elf-objcopy -I binary -O elf32-i386 -B i386 \
	--rename-section .data=.text payload.tmp payload.o
ia16-elf-gcc -mcmodel=medium main.o payload.o
/usr/lib/x86_64-linux-gnu/gcc/ia16-elf/6.3.0/../../../../../ia16-elf/bin/ld: a.out section `.text' will not fit in region `csegvma'
/usr/lib/x86_64-linux-gnu/gcc/ia16-elf/6.3.0/../../../../../ia16-elf/bin/ld: Error: too large for a small-model .exe file.

[tkchia]

Hello @stsp,

The medium memory model definitely works in gcc-ia16. I am not sure exactly what high-level goal you are trying to accomplish, but you are probably doing it wrong.

Thank you!

[asiekierka]

Trying to put a single 100K payload into a single .text section (which means a single segment, which means a size limit of 64K) is not going to end well.

If you observe the -mcmodel=medium main.o generated by the C compiler (objdump is your friend), you'll notice that the C compiler generates special .fartext sections to work around this, as well as special symbols and relocations that handle segmentation. objcopy doesn't implement this scaffolding, and as such will cause additional headaches.

The medium memory model in itself certainly does work - I am using it right this second, as I'm developing a program with gcc-ia16. The issue is that (a) you're trying to put a single 100K symbol in a 64K segment, which is not going to work; (b) you're using an object generation tool (that is objcopy) which does not emit segment information in the object file; all symbols in .text will end up in one 64K segment, likewise .data; you need additional sections for additional symbols. Analyzing the output of gcc-ia16 with objdump will get you further along, if you want to go down that path - I recommend just using a bin2c-style tool.

[stsp]

(a) you're trying to put a single 100K symbol in a 64K segment, which is not going to work

OK, thanks. I tried smaller files with objcopy - 2 files 60K each, one in .text, one in .text2. Unfortunately linker script seems to discard .text2, so no luck with objcopy. Thanks about the bin2c hint, will try.

[asiekierka]

Yeah, it's not just about defining a ".text2" section - you'd need to add it to the linker file. The linker files bundled with build-ia16 handle section names generated by gcc-ia16.

https://github.com/tkchia/build-ia16/blob/master/elf16-writeup.md - this writeup isn't exactly meant to be an explanation of the system, but it might give you a starting point.

[stsp]

what high-level goal you are trying to accomplish, but you are probably doing it wrong.

Yes, you are right. All I need is to add a payload to an exe file. I thought I can do that with medium model, but I don't have to. And in fact shouldn't, small model is perfectly enough. So is there some linker trick to add a non-loadable payload to an exe?

[tkchia]

Hello @stsp,

So is there some linker trick to add a non-loadable payload to an exe?

Let me see if I can add something to newlib-ia16 's linker scripts to allow adding a payload or overlay at the end. Thank you!

[stsp]

Thanks! Maybe you can allow objcopy to do the trick after all? If you introduce some .overlay section, then objcopy can use that.

[tkchia]

Hello @stsp,

I patched (https://github.com/tkchia/newlib-ia16/commit/6480c844ae357d6d566a46d0b2e9f10bf00af0d8) the newlib-ia16 linker scripts to recognize input section names .tail, .tail.whatever, .ov, or .ov.whatever. (If you use the whatever parts, the .tail.whatever and .ov.whatever sections will be sorted by name.)

The sections' contents will be placed after the end of the .exe proper as indicated by the MZ header's .e_cblp and .e_cp fields.

If your program contains such payload sections, then it should be able to find them by opening argv[0] and parsing the MZ header. (Well, on MS-DOS 3 and above, anyway. MS-DOS 2.x does not pass the program name in the environment segment.)

The new newlib-ia16 should be available on my PPA soon, in an hour or so.

Thank you!

[stsp]

Thanks, that's what I am looking for. So am I right that I'll have to employ objcopy after all, and create a binary with that section names?

[tkchia]

Hello @stsp,

Well, for creating the .o file, there are several ways:

• You can use objcopy as before.
• Or, you can use some sort of bin2c tool (as @asiekierka mentioned).
• Or, you can write a .s or .S file that uses .incbin to slurp in the binary content, then assemble that (remember to track dependencies properly).

Thank you!

[tkchia]

@stsp : do let me know when you are OK with closing this issue. Thank you!

[stsp]

I am still getting this:

ia16-elf-objcopy -I binary -O elf32-i386 -B i386 \
	--rename-section .data=.tail payload.tmp payload.o
ia16-elf-gcc -mcmodel=medium main.o payload.o
main.o: in function `main':
(.text+0x8): relocation truncated to fit: R_386_16 against `_binary_payload_tmp_size'
(.text+0xc): relocation truncated to fit: R_386_16 against `_binary_payload_tmp_end'

So may I assume some work is needed on an objcopy side?

[tkchia]

Hello @stsp,

No, no, no, no, no, this will definitely not work:

int main()
{
    printf("start %p end %p size %x\n",
      _binary_payload_tmp_end,
      _binary_payload_tmp_size,
      _binary_payload_tmp_start);
    return 0;
}

I have already outlined what will work. You might want to read what I write again.

Open argv[0], parse the MZ header, then read the payload.

Thank you!

[stsp]

Ah I didn't know these lines are a source of a failure. Leaving just the "start" mark makes it to work. By the way can I deduce the offset in an exe file from that "start" marker?

[tkchia]

Hello @stsp,

No unfortunately. The _binary_payload_tmp_start marker will most likely be wrong — it will simply be set to 0 (IIRC). Best to obtain the payload offset from the MZ header.

Thank you!

[stsp]

start 0xff8c Yep, doesn't make much sense (too large value).

[tkchia]

Hello @stsp,

If it is any comfort: you only need to care about (and read) the first 3 fields of the MZ header: magic number (.e_magic), number of bytes on last page (.e_cblp), and number of pages (.e_cp). And remember to handle the special case where .e_cblp = 0. Thank you!

[stsp]

Hmm, this doesn't work with -mdosx.

ia16-elf-gcc -mcmodel=small -mdosx main.o payload.o -li86 -o main.exe
/usr/lib/x86_64-linux-gnu/gcc/ia16-elf/6.3.0/../../../../../ia16-elf/bin/ld.gold: warning: script places BSS section in the middle of a LOAD segment; space will be allocated in the file

And payload is not appended.