elishacloud
Broken characters while translating to PT-BR
Hey, folks.
First of all, great job on the Enhanced Edition! It's amazing how much the community can still get out of this game almost 20 years later!
I have been trying to make a new Brazilian Portuguese translation, as the available one has quite a few mistakes. I did some research and found similar issues for other languages, but they didn't solve what's happening on my end.
I'm using the Ultimate SH-MES Recompilator to extract the English .mes files and translate them. I did try using the SH2MSGConvert tool, but I got this error when opening it on the CMD Prompt app:
usage: sh2msg.exe [-h] [--file-list FILE_LIST] [--output OUTPUT]
[--table TABLE] [--table-jap] [--raw-mode]
[FILE [FILE ...]]
sh2msg.exe: error: one of the arguments FILE --file-list/-l is required
Anyway, I translated the Options file as a test, and it came out all wrong:
I did look at the .tga and .bin font files issue, but they seem to include all special characters used in that screenshot:
- ç
- ã // õ
- ú
(If you want to be more specific, the Portuguese language can use the acute, grave, circumflex, and tilde accents, but only on vowels -- with ç being the exception.)
I did try grabbing the .exe from another fan translation and tested one line in the Control Options (Opções de Controle). That one worked well. But then I tried the line when James encounters the first save point in the forest and that was a bunch of trial and error, switching between the EE .exe and the PT-BR .exe. Eventually, I managed to extract the broken line with the SH-MES Recompilator, but that doesn't work often. It feels super finicky. It extracts the files exactly as I translated them, and I have no idea how I got it to extract with broken characters the one time.
It's weird because if the issue was with the font files, it would have never worked -- unless I'm missing something! At some point, both of the .exes displayed the correct characters. It was only when I decided to translate the whole menu that I couldn't manage to get it working again.
Could it be the extract tool I'm using? If so, what am I missing to make SH2MSGConvert work?
Update: Fixed the broken character issue by changing Notepad++'s encoding to ANSI.
However, I can't seem to change the Main Menu options to Portuguese using the EE's .exe. Is that hard coded?
Hi there,
To start, thanks for doing a lot of research on your own to figure out the general tools/processes used to make a new language pack.
@ilducci is one of the creators of sh2msgconvert. He can help with any troubleshooting for the tool, if you run into any more issues.
@renan-de-souza created the most recent Portuguese language pack. Perhaps he could collaborate with you to help for a revised PT pack.
For the main menu: This is a texture image (for exe v1.0/1.1) and not live text (as it is for EU exe). I would need to make this main menu image for you. I won't be able to do this for a few days, but in the meantime, check out this post to know what I need from you and to also see the text length limitations:
- https://github.com/elishacloud/Silent-Hill-2-Enhancements/issues/234#issuecomment-593582332
- https://github.com/elishacloud/Silent-Hill-2-Enhancements/issues/234#issuecomment-594145904
Of course! Here are the words you need:
CARREGAR
CONTINUAR
NOVO JOGO
OPÇÕES
VÍDEOS
SAIR
NÍVEL DE COMBATE
NÍVEL DE ENIGMAS
DIFÍCIL
NORMAL
FÁCIL
INICIANTE
Cenário Principal: CARTA DO CÉU SILENCIOSO
Sub-Cenário: NASCIDA DE UM DESEJO
@IlDucci I'd appreciate any help running the tool, as I don't have experience running from cmd!
@renan-de-souza Definitely open to collaborate if you want!
@RobertoSAG So the best way to use the tool is by writing a couple of batch files. These are text files with commands that have a .BAT extension.
So put in the same folder the sh2msg executable or folder, depending on your choice (I used the python files, having Python 3 installed), another folder with the files you want to extract, and write a text file with this:
FOR %%F IN (folderwithMESfileshere\*.mes) DO python -m sh2msg %%F
For the standalone executable, it should be like this:
FOR %%F IN (folderwithMESfileshere\*.mes) DO sh2msg %%F
Rename the text file extension to .BAT and double click it. If everything went well, you should have a bunch of TXT files in the same folder as the MES files. If you want to reimport the text files into the MES ones, just do another copy of the batch file and, inside, replace the *.mes part with *.txt.
@RobertoSAG So the best way to use the tool is by writing a couple of batch files. These are text files with commands that have a .BAT extension.
So put in the same folder the sh2msg executable or folder, depending on your choice (I used the python files, having Python 3 installed), another folder with the files you want to extract, and write a text file with this:
FOR %%F IN (folderwithMESfileshere\*.mes) DO python -m sh2msg %%FFor the standalone executable, it should be like this:
FOR %%F IN (folderwithMESfileshere\*.mes) DO sh2msg %%FRename the text file extension to .BAT and double click it. If everything went well, you should have a bunch of TXT files in the same folder as the MES files. If you want to reimport the text files into the MES ones, just do another copy of the batch file and, inside, replace the
*.mespart with*.txt.
Hey, @IlDucci, thanks for your reply! Most of the files worked well, except for a few that displayed these error codes (I shortened the filepath before putting it here):
sh2msg-win-amd64>sh2msg FILE_LIST\option_msg_e.mes
[ERROR Dump .mes] ERROR sh2msg-win-amd64\FILE_LIST\option_msg_e.mes Unknown bytes 6D address 6100
sh2msg-win-amd64>sh2msg FILE_LIST\option_msg_f.mes
[ERROR Dump .mes] ERROR sh2msg-win-amd64\FILE_LIST\option_msg_f.mes Unknown bytes 80 address 691
sh2msg-win-amd64>sh2msg FILE_LIST\option_msg_s.mes
[ERROR Dump .mes] sh2msg-win-amd64\FILE_LIST\option_msg_s.mes Unknown bytes 6D address 6245
sh2msg-win-amd64>sh2msg FILE_LIST\result_msg_s.mes
[ERROR Dump .mes] sh2msg-win-amd64\FILE_LIST\result_msg_s.mes Unknown bytes 6D address 299
sh2msg-win-amd64>sh2msg FILE_LIST\stage_x_heaven_msg_f.mes
[ERROR Dump .mes] ERROR sh2msg-win-amd64\FILE_LIST\stage_x_heaven_msg_f.mes Unknown bytes
80 address 846
I wouldn't miss most of those, except for the option_msg_e.mes one. Still, do you have any idea why it's happening? I'm using the standalone executable.
Hmmm, that is odd. That means that the tool is not recognising some encoded bytes, which it should (the non-executable version was thoroughly tested, but the EXE gave some trouble). Please make sure you're using the latest version of the EXE file, and if it's still failing, use the scripts version (just download the sh2msg folder, paste it next to the folder with the MES files, and use the commands I've said).
I'm going to tag @Infrid just in case.
Hmmm, that is odd. That means that the tool is not recognising some encoded bytes, which it should (the non-executable version was thoroughly tested, but the EXE gave some trouble). Please make sure you're using the latest version of the EXE file, and if it's still failing, use the scripts version (just download the sh2msg folder, paste it next to the folder with the MES files, and use the commands I've said).
I'm going to tag @Infrid just in case.
@IlDucci I installed Python3 (latest version - 3.9.0), downloaded the non-executable version, and tried using a new .bat file. It did detect all .mes files, and it seemed to go through all of them, but it didn't create any .txt out of them. Any ideas?
@Polymega I also have another question for you, if you don't mind! What's the practical difference between "Data" and "Save" on the Save Menu, below the table? I want to make sure the translation fits with that, but the numbers aren't the same if the scenario is different. I understand that part, but not the logic behind it.
Data is the save file slot number. If you scroll through all your save files, you'll see the data number is in sequential order as you scroll through your save list.
Think of it as Save File 01, Save File 02, etc.
And Save is the total of how many times the player has saved their game progress so far in that particular playthrough.
Data is the save file slot number. If you scroll through all your save files, you'll see the data number is in sequential order as you scroll through your save list.
Think of it as Save File 01, Save File 02, etc.
And Save is the total of how many times the player has saved their game progress so far in that particular playthrough.
Thank you so much!
Hello!
The problem in this case is not the tool, but in the table it uses for converting text files to .mes, let me explain a bit better.
Every file is a sequence of bytes, the meaning of the bytes is up to the programmer/user, the computer doesn't know what they mean. When you open a file in notepad and you set the encoding in ANSI, you are telling to the computer that the number 0x41 is equal to the character "A", 0x42 is equal 0x42, etc etc.
When you use the "Encoding" menu in notepad, you are basically telling to the computer to load a table where 41=A and 42=B, since that encoding is a standard you don't have to define the meaning of bytes upfront. Computers already know it!
This is not true for Silent Hill 2! The computer doesn't know how to read those files, the programmers during the development of the game invented an entire new encoding, I will not explain why for keeping the explanation short :)
That's why we created sh2msg tool, it's not a text editor but a converter that will translate an encoding that every computer knows to the encoding made for SH2.
Look at this file
https://gitlab.com/sh2msgconvert/sh2msgconvert/-/blob/master/sh2msg/table/data/table.txt
there we define how the SH2 encoding works, for example the line
21=A
means the game uses 0x21 to represent the char A.
If you read the entire table, you will not find an entry where 80 or 6d are defined, the tool doesn't know how to convert those values, how can you fix that?
You can define your table and feed that to the tool, this feature is vital for other languages where the alphabet is totally different from what the game use and the translator has to re-define all the glyphs.
Copy that table.txt to your computer and add the lines
80=X 6D=Y
where X and Y are the characters that are missing and use it with the tool with the --table=table.txt
This method will work and we can add them to the tool for everyone if those are native to the game!
Let me know if you manage to write your table.txt. By the way the tool works really well with utf8 files (and supports BOM too), you can set that encoding in notepad++ and leave it.
@RobertoSAG I know it's not exactly the same as the translations you've given me, but I looked at my files and apparently already made a Portuguese translation for the high resolution main menu:
...chances are I made this for @renan-de-souza. It might be good to see if he'll respond, to perhaps help ensure there's one "global" Portuguese translation pack out there for people to use, instead of multiple variations.
I can still remake this high resolution main menu using your translations instead. And if I do, I'll just delete the previous translation for it, as I'd prefer not to have variations of the same language for the menu if it can be helped.
Hey, @Infrid, thank you for that explanation. It seems like I couldn't convert some of the English .mes files because I was using the ANSI encoding from another tool. I successfully converted all of them now. It did give errors on some of the Spanish and French files I'm using as a reference when needed, but that's not a big deal for me. I didn't need to include new characters on the table!
To be clear, I didn't change anything on the Spanish and French files, so I'm not sure why they gave errors. Hopefully, you can figure that out if needed!
@Polymega I appreciate that! I noticed that it includes some words I didn't translate for you, and they're still in English. I imagine that's because they're never shown on the main menu? I'm not sure what "BATTLE" does here, for example, and I'm assuming "INSTALL" is for the Xbox version. I'm also using a couple of different words for some of them. For instance, I changed "NÍVEL DE AÇÃO" to "NÍVEL DE COMBATE" to better represent that this is the combat difficulty (as "action" can be a bit vague) and "CENAS" to "VÍDEOS" (as it includes both cutscenes and trailers). I do like the idea of changing "Cenário" to "História" to follow the same logic.
Having said that, if you want all of them translated for the sake of cohesiveness (and assuming "BATTLE" is just... "battle"), here are all of them:
CARREGAR
CONTINUAR
NOVO JOGO
OPÇÕES
PRISÃO
APARTAMENTO
HOTEL
HOSPITAL
PRÓLOGO
BATALHA
SAIR
NÍVEL DE COMBATE
DIFÍCIL
NORMAL
FÁCIL
INICIANTE
NÍVEL DE ENIGMAS
DIFÍCIL
NORMAL
FÁCIL
INICIANTE
INSTALAR
VÍDEOS
História Principal: A CARTA DO CÉU SILENCIOSO
História Secundária: NASCIDA DE UM DESEJO
On another note, I noticed the "message" folder from "sh2e" doesn't include these files:
config_pc_msg_e.mes
stage_apart_e3fe_msg_e.mes
stage_hospital_1fe_b_msg_e.mes
stage_hospital_pass_msg_e.mes
stage_hotel_3f_b_msg_e.mes
stage_hotel_bf_b_msg_e.mes
stage_sample_msg_e.mes
stage_tgs_trial_msg_e.mes
stage_to_heaven_msg_e.mes
Is it okay to not translate these? Or should I copy them to the folder and translate? It seems clear the TGS one is just for a trial, but I'm especially curious about the others.
To be clear, I didn't change anything on the Spanish and French files, so I'm not sure why they gave errors. Hopefully, you can figure that out if needed!
I just redumped the Spanish files from the sh2ee project and they were dumped correctly. Are you sure the French and Spanish assets you used came from either the original, unmodified game or from the Enhanced Edition's texts?
On another note, I noticed the "message" folder from "sh2e" doesn't include these files:
config_pc_msg_e.mes stage_apart_e3fe_msg_e.mes stage_hospital_1fe_b_msg_e.mes stage_hospital_pass_msg_e.mes stage_hotel_3f_b_msg_e.mes stage_hotel_bf_b_msg_e.mes stage_sample_msg_e.mes stage_tgs_trial_msg_e.mes stage_to_heaven_msg_e.mesIs it okay to not translate these? Or should I copy them to the folder and translate? It seems clear the TGS one is just for a trial, but I'm especially curious about the others.
Those files are probably untouched by the EE. config_pc, tgs_trial and sample are leftovers that can be avoided. The rest should be translated. (I think you can also ignore letter_msg_e, as w0/w1/w2 are the ones that are indeed used).
To be clear, I didn't change anything on the Spanish and French files, so I'm not sure why they gave errors. Hopefully, you can figure that out if needed!
I just redumped the Spanish files from the sh2ee project and they were dumped correctly. Are you sure the French and Spanish assets you used came from either the original, unmodified game or from the Enhanced Edition's texts?
I tried from a backup of the EE files and it worked correctly. My guess is that Notepad++ changed the encoding to ANSI when I was using the other converter. I will keep using yours though, as it's the one recommended by the EE team. Thank you!
On another note, I noticed the "message" folder from "sh2e" doesn't include these files:
config_pc_msg_e.mes stage_apart_e3fe_msg_e.mes stage_hospital_1fe_b_msg_e.mes stage_hospital_pass_msg_e.mes stage_hotel_3f_b_msg_e.mes stage_hotel_bf_b_msg_e.mes stage_sample_msg_e.mes stage_tgs_trial_msg_e.mes stage_to_heaven_msg_e.mesIs it okay to not translate these? Or should I copy them to the folder and translate? It seems clear the TGS one is just for a trial, but I'm especially curious about the others.
Those files are probably untouched by the EE. config_pc, tgs_trial and sample are leftovers that can be avoided. The rest should be translated. (I think you can also ignore letter_msg_e, as w0/w1/w2 are the ones that are indeed used).
Perfect! Thank you!
@Polymega Do you happen to have the save files for all riddle difficulties? I'd love to make sure every memo is working as intended.
Do you happen to have the save files for all riddle difficulties?
Sure, completely replace your \data\save\Folder 01\ folder with this new Folder 01 folder: Folder 01.zip
The save files are as followed:
- Easy Riddle
- Normal Riddle
- Hard Riddle
- Extra Riddle
Also, here is the revised Portuguese main menu text: start01p.zip
Be mindful when you translate save file locations. While on the save menu, press the left or right arrows to cycle through save file details. You'll notice the detail of the save date/time is a long string. Try to make sure your translated save file locations do not go past this area:
Lastly, in regards to the save screen, it has some rather strange/silly limitations. You need to ensure the verbiage at the top (on the save folder's "tab") is less than 11 characters. And also ensure the "delete file with backspace" message is kept very short (I believe less than 27 characters).
If you don't do these things, certain characters on the save screen will disappear or "flicker" as you scroll through the save list. More information on this can be read here: https://github.com/elishacloud/Silent-Hill-2-Enhancements/issues/111#issuecomment-568934361
Do you happen to have the save files for all riddle difficulties?
Sure, completely replace your
\data\save\Folder 01\folder with this newFolder 01folder: Folder 01.zipThe save files are as followed:
- Easy Riddle
- Normal Riddle
- Hard Riddle
- Extra Riddle
Also, here is the revised Portuguese main menu text: start01p.zip
Be mindful when you translate save file locations. While on the save menu, press the left or right arrows to cycle through save file details. You'll notice the detail of the save date/time is a long string. Try to make sure your translated save file locations do not go past this area:
Lastly, in regards to the save screen, it has some rather strange/silly limitations. You need to ensure the verbiage at the top (on the save folder's "tab") is less than 11 characters. And also ensure the "delete file with backspace" message is kept very short (I believe less than 27 characters).
If you don't do these things, certain characters on the save screen will disappear or "flicker" as you scroll through the save list. More information on this can be read here: #111 (comment)
Thank you so much, @Polymega! I managed to keep the "Save Folder" part to 12 characters ("Dados Salvos"). Is that okay? It doesn't seem to flicker on my end. If not, I could maybe change it to simply "Arquivos" (meaning "Files"). I changed "Delete file with Backspace" to "Apagar com Backspace" so that should be okay.
I took screenshots at 640x480 to see what you think. The longest one ended up being the "Escadaria/Porão" one, but I can keep it to just "Porão" ("Basement") if necessary.
By the way, about the save files on all riddle difficulties, I was wondering if you had them in a way I could check the Memos menu on all difficulties? Or if there's a method I could check all of them in-game. The ones I've found only have them at Easy, unfortunately.
Finally, the menu works great! I renamed it to "start01e.tex" (making a backup of the original file) since the game doesn't have an official Portuguese language option. I assume I can only replace it like that instead of adding another option?
By the way, about the save files on all riddle difficulties, I was wondering if you had them in a way I could check the Memos menu on all difficulties? Or if there's a method I could check all of them in-game. The ones I've found only have them at Easy, unfortunately.
Unfortunately, these saves, for some reason, were from a game that was done without checking any Memo that wasn't required by the plot, kinda like a speedrun. You'll have to beat the game on new save files (I use the plural because some Memos change depending on Action/Enigma difficulties) in order to have that quick access to every file.
I managed to keep the "Save Folder" part to 12 characters ("Dados Salvos"). Is that okay?
It may be! The best thing to do is to test as you're doing. Keep scrolling up and down on the save file list to see if the "Dados Salvos" text gets cut off any. This bug is very incosistent.
The longest one ended up being the "Escadaria/Porão" one, but I can keep it to just "Porão" ("Basement") if necessary.
This is your translation pack, so translate to what you think is best/appropriate! I just wanted to give some tips about overlapping text in the save menu, is all. :) But yes, your screenshots look good!
By the way, about the save files on all riddle difficulties, I was wondering if you had them in a way I could check the Memos menu on all difficulties?
Unfortunately, I do not have save files on all Riddle levels for all the memos. If you have ever used a memory editor, you can use memory address 01DBBFF5 (byte) to change it in real-time. Although, I'm not sure if the game will correctly/automatically switch the puzzles to the new difficulty in real time...
01DBBFF5 (byte) Riddle Level
0 = Easy
1 = Normal
2 = Hard
3 = Extra
Finally, the menu works great! I renamed it to "start01e.tex" (making a backup of the original file) since the game doesn't have an official Portuguese language option. I assume I can only replace it like that instead of adding another option?
Yes! That's correct. Replace it with any language you'd like. So if you want to replace the English pack with Portuguese, you'd replace start01e.tex as you've done.
I never tried using one, but I can try! I'll report back later.
I also just thought I could temporarily replace the Easy text with the ones for other difficulties for testing purposes, which might be an even simpler solution.
This is what I tell everyone working on the .mes files: Test, test, then test some more all of your changes to make sure they display correctly and as you intended in the game!
And I'm sure you figured this out, but every memo/riddle text is stored in at least two .mes files:
One at the location it's found (such as an apartment .mes file) and the other in the memo_msg_*.mes file.
Finally, the menu works great! I renamed it to "start01e.tex" (making a backup of the original file) since the game doesn't have an official Portuguese language option. I assume I can only replace it like that instead of adding another option?
Yes! That's correct. Replace it with any language you'd like. So if you want to replace the English pack with Portuguese, you'd replace
start01e.texas you've done.
There is actually code to handle this. We can make a separate file for each language , if desired.
The tricky thing with adding new languages without replacing others is for the main menu, which you linked the code to, but also telling the game to read example_*.mes (with the asterisk being a new language letter), adding the new language as a selectable option in the Options menu, and assigning a new byte for it in the language select memory address (example: 0 = Japanese, 1 = English, etc).
It's probably best to replace all of another's language files as RobertoSAG is doing for that reason.
I know a lot of Brazilian folks still want to have the English option available, so I thought about replacing the Spanish files with the English files (including the main menu textures).
The only things I can't find how to change are the in-game Pause Menu and the instructions to change audio settings inside Windows. Where can I change these?
Those can be found in the d3d8.res file. Open that file through a text editor to change it.
Perfect! I think I have everything I need. I'll let you know if I need anything else.
@Polymega what about the credits? Is there a way to translate them as well, and possibly including my name after I'm done translating everything?
@elishacloud If you don't mind me asking either, is there any other QOL update planned that can change the text somewhere? Just in case I might need to make some backup files beforehand.
At this point, I don't have any QOL changes planned for the text or translations. I don't think we want to make any changes here because that would require us to change all languages, which we don't really want to do.