resinator icon indicating copy to clipboard operation
resinator copied to clipboard
verified publisher icon squeek502

Still not able to get proper CodePage value for resources in a final exe/dll file

Open gh-Destiny opened this issue 7 months ago • 5 comments

I decided to try to choose your product so that I can always confidently get multi-language final products in compiled resources compiled exe/dll files from Visual Studio 2019. As I understood it - this is requiring that resources that use a text component always match the correct code page. And then these strings will always be displayed correctly on the screen / faithfully transmitted by client-server communication.

What was done: the resource file (*.rc file) was formatted and saved in UTF-8+BOM encoding, as the most common and adequate for use among any development teams/just regular people even for simple communication. As well as all other project files. Knowing in advance that the resource compiler (rc.exe from Microsoft) is prone to problems when building multilingual projects (in particular from the information on your blog), the following was written at the beginning of the rc file: #pragma code_page(65001) The project was built, launched, and checked. Everything seems to work. But this is on our platform/comps/OS/.... We sent it to a person on the other side of the Earth, and nothing good came out of it. The text became unreadable. Then we checked the status of the CodePage field of the IMAGE_RESOURCE_DATA_ENTRY structure from the well-known description of the PE/COFF format and saw that zero 0x0000 (0) (ANSI - Default) was set for all resources compiled and stored in my EXE file.

Ok, so I tried to use YOU utility: resinator.exe /c 65001 /fo MyEXEtool.res ..\MyEXEtool.rc It seems that the file I was looking for appeared on the disk and there were no errors when creating it. Then I further linked this file with the object files of my project and received the final EXE. It was definitely a new file. I ran it, checked it, and everything was fine. On my computer, OS, etc... BUT a new check with the special utility showed that nothing has changed in terms of the value of the CodePage field.... Questions: Did I do something wrong? is your utility still unable to specify a value for this field? is Microsoft's linker incompatible with the final res file obtained from your utility? Or something else?

I can physically see that this is possible in principle, because I see different real files where this field is filled with a valid constant, and not just a default zero. Here's the question - how to ensure that it always works like this/generates the final rc/res/obj/exe/dll/...

gh-Destiny avatar May 14 '25 20:05 gh-Destiny

Code page information is not stored in the .res file at all (here's the structure used for each resource).

IMAGE_RESOURCE_DATA_ENTRY is written by cvtres.exe and/or link.exe, but I'm not sure if they ever write anything other than 0 for CodePage.

Can you provide steps to produce a IMAGE_RESOURCE_DATA_ENTRY with a CodePage that is not 0 using the Microsoft compiler tools?

squeek502 avatar May 14 '25 21:05 squeek502

You know, your answer makes me even more confused.... According to you, it turns out that if we just always use the full set of VisualStudio utilities (development environment), then we will always get 0 for this field? And you have NOT ever seen the opposite in your analyses of the behavior of these tools? That's why you're asking me about the steps to get non-zero values?

BUT I don't have such steps - I just have ready-made exe/dll files in my memory for which I had previously SEEN non-zero values of this field in the resource viewer.... That is, someone was able to do this somehow. Here is a close example - https://www.upload.ee/files/18100237/mediacreationtool.exe.html And since I saw them, and since I couldn't get something like this on my side, I tried to find explanations for it on the Internet. And so I came across your article, in which you seem to have described a large number of glitches typical of the standard rc.exe and it seems like you tried to reduce them to zero in your product. And it seems that in the same article you write about the same #pragma construction, which should have fixed the situation with this zero. But it turns out that this is not the case? this is NOT a panacea?

P.S. If we could ask directly Raymond Chen - I believe we would instantly get all answers on these questions... But even he already said: https://devblogs.microsoft.com/oldnewthing/20190607-00/?p=102569 that we should use #pragma construction....

gh-Destiny avatar May 15 '25 10:05 gh-Destiny

There are two distinct steps that happen with resource compilation:

  • The .rc file is compiled into a .res file
    • This is what resinator and rc.exe do (and resinator almost always makes .res files that are byte-for-byte identical to what rc.exe creates)
  • The .res file is compiled into a COFF .obj file (and then linked into the final binary)
    • This is what cvtres.exe does (and link.exe calls out to cvtres.exe if you pass a .res file to it)

I'm saying that the .res file does not contain information about the code page of any of the resources within it. Therefore, cvtres.exe cannot know what code page a resource might have; the code page information from the .rc file is lost when it is compiled into a .res file.

So, I'm not saying it's impossible for cvtres.exe/link.exe to write a non-zero value for CodePage in IMAGE_RESOURCE_DATA_ENTRY, I'm just unfamiliar with when that can happen, and AFAIK it would probably have to make a guess.

Beyond that, I'd also ask why the CodePage field in IMAGE_RESOURCE_DATA_ENTRY is important for your use case. What changes if it is or is not set to the proper value? Can't you just load the resource and treat it as the code page you know it to be?

squeek502 avatar May 15 '25 19:05 squeek502

Until I found out about the presence of the #pragma command when I was viewing running dialogs, the entire text was in obscure characters. And the first thing - the Internet gave me when trying to find an answer to a logical question about this - is that the CodePage field is responsible for this. I quickly checked its value and with big surprise I was taken aback that I had only zeros in this field. Meaningless zeros. After that, I began to search for other files - and find them - where there was NOT ZERO in this field. I sent you such one, didn't you check it? You have checked that there are non zeros CodePage values there, right? For All resources! Then there were other articles/blogs, including yours. And that's when I seemed to have managed to achieve the correct display of text symbols in dialogues by enabling the #pragma construction. But the CodePage field was still 0. I just can't figure this out if I don't fully understand whether this field impacts something real - or if it's a complete relic from the 80s, from where the resource compiler from M$ originates.

gh-Destiny avatar May 15 '25 19:05 gh-Destiny

I understand that it's possible for that field to be set, and that it might be common for that field to be set in Microsoft binaries. I don't currently know:

  • how it's possible for that field to be set when using the publicly provided compiler tools (rc.exe, cvtres.exe, link.exe) with documented options
  • what it affects if it is set to the wrong value
  • how cvtres.exe or link.exe could know what the right value for the CodePage of a resource is

resinator verifiably does the same thing as rc.exe (you can check this with your .rc file by compiling it with resinator and rc.exe with the same options and verifying that the resulting .res files are byte-for-byte identical), so what happens beyond that is somewhat out-of-scope.

However, resinator will provide a cvtres.exe implementation as well, so this topic is relevant for that use-case and I will be looking into it more.

squeek502 avatar May 15 '25 20:05 squeek502