Adjustments to Accommodate Japanese Language/2-Byte Languages

[Polymega]

Hi Elisha,

We're in the process of restoring the Japanese language for the game and came across two items that are affecting this work. I'd like to run these by you, if I may.

The first we talked about previously, and that's adding support for the Japanese language with the fix to show available save space in units other than KB. The fix is correctly showing save space in different measurements, but the baked-in "KB" part remains in the Japanese language:

The other thing I'd like to bring up is how d3d8.res is currently handled. Now that we have the ability to restore the Japanese language, including its characters in the d3d8.res file has proved tricky. First, let's show how we'd ideally want the d3d8.res to look:

// Japanese

PAUSE
\hResume
\hSave Game
\hLoad Game
\hOptions
\hQuit
タイトル画面に戻りますか？
はい
いいえ
\cサイレントヒル 2のディスクを挿入して下さい
\cポーズ
\c異なるディスクです
\cゲームパッドが未接続です
\cディスクが見つかりません
\c再度お試し下さい
\c\hNow loading
\cお待ちください…
\cディスクを確認中
\cタイトル画面に戻りますか？
\cはい　
\c　いいえ
Button %d
セーブしました
セーブできません
クイックセーブ不可
d3d8.iniファイルを設定
Windows側でのみ音響調整可

// English

\hPAUSED
\hResume
\hSave Game
\hLoad Game
\hOptions
\hQuit
\hExit to Main Menu?
\hYes
\hNo
\c\hPlease insert Silent Hill 2 disc
\c\hPaused
\c\hWrong disc
\c\hController not connected
\c\hDisc not inserted
\c\hDisc tray open
\c\hNow loading
\c\hPlease wait
\c\hChecking disc
\c\hExit to main menu?
\c\hYes
\c\hNo
Button %d
\hGAME SAVED
\hCAN'T SAVE
\hNO QUICK SAVE
\hConfigure in d3d8.ini file
\hConfigure in Windows Sound Settings

// French

\hPAUSE
\hReprendre
\hSauvegarder partie
\hCharger partie
\hOptions
\hQuitter
\hQuitter vers le menu principal?
\hOui
\hNon
\c\hVeuillez insérer le disque de Silent Hill 2
\c\hEn pause
\c\hMauvais disque
\c\hLa  manette n'est pas connectée
\c\hDisque non inséré
\c\hLecteur de disque ouvert
\c\hChargement
\c\hVeuillez patienter
\c\hVérification du disque
\c\hQuitter vers le menu principal?
\c\hOui
\c\hNon
Bouton %d
\hPARTIE SAUVEGARDEE
\hSAUVEGARDE IMPOSSIBLE
\hAUCUNE SAUVEGARDE RAPIDE
\hRégler la résolution dans d3d8.ini
\hChanger ce réglage dans Windows

// German

\hPAUSE
\hWeiter
\hSpiel speichern
\hSpiel laden
\hOptionen
\hBeenden
\hZurück zum Hauptmenü?
\hJa
\hNein
\c\hBitte Silent Hill 2 Disc einlegen
\c\hPausiert
\c\hFalsche CD
\c\hController nicht angeschlossen
\c\hDisc nicht eingelegt
\c\hDisc-Fach geöffnet
\c\hWird jetzt geladen
\c\hBitte warten
\c\hDisc wird überprüft
\c\hZurück zum Hauptmenü?
\c\hJa
\c\hNein
Taste %d
\hSPIEL GESPEICHERT
\hKANN NICHT SPEICHERN
\hKEINE SCHNELLSPEICHERUNG
\hAuflösung in d3d8.ini einstellen
\hAudioeinst. in Windows vornehmen

// Italian

\hPAUSA
\hRiprendi
\hSalva partita
\hCarica partita
\hOpzioni
\hAbbandona
\hTorna al menu principale?
\hSì
\hNo
\c\hSi prega di inserire il disco Silent Hill 2
\c\hIn pausa
\c\hDisco sbagliato
\c\hController non connesso
\c\hDisc not inserted
\c\hDisco non inserito
\c\hIn caricamento
\c\hAttendere prego
\c\hControllo del disco
\c\hTorna al menu principale?
\c\hSì
\c\hNo
Tasto %d
\hPARTITA SALVATA
\hSALVATAGGIO NON RIUSCITO
\hNESSUN SALVATAGGIO VELOCE
\hImposta la risoluzione nel file d3d8.ini
\hConfigura attraverso Windows

// Spanish

\hPAUSA
\hReanudar
\hGuardar partida
\hCargar partida
\hOpciones
\hSalir
\h¿Salir al menú principal?
\hSí
\hNo
\c\hIntroduzca el disco de Silent Hill 2
\c\hPausa
\c\hDisco incorrecto
\c\hMando no conectado
\c\hDisco no insertado
\c\hBandeja de disco abierta
\c\hCargando
\c\hEspere, por favor...
\c\hComprobando disco
\c\h¿Salir al menú principal?
\c\hSí
\c\hNo
Botón %d
\hPARTIDA GUARDADA
\hNO SE PUEDE GUARDAR
\hNO HAY GUARDADO RÁPIDO
\hCambiar resolución en d3d8.ini
\hConfigurar a través de Windows

In order to save out special Latin characters (such as "ó" or "ú"), the d3d8.res file needs to be encoded in a certain way. But when encoded this way, the Japanese characters are no longer supported. If we encode it to support Japanese characters, the special Latin characters are no longer supported.

I've been speaking with @IlDucci about this, who is our translator/encoder specialist, and we're not sure how to approach this other than the possibility of having each respective language be saved out into their own files. That way, each file could be encoded differently. Do you have any ideas/thoughts for this?

A nice thing about this extra effort to support the Japanese language is that it opens the door for Chinese, Korean, and other 2-byte character language fans to make translation packs without any hurdles/compromises moving forward.

[elishacloud]

adding support for the Japanese language with the fix to show available save space in units other than KB.

Yes, I have this on my list but have not had time to work on it yet. I will egt to it soon.

But when encoded this way, the Japanese characters are no longer supported.

What encoding are we attempting to use? Is there some reason we cannot use UTF-8 encoding? That should support all of the characters for all of the languages.

An other option would be to store each of the languages in a different file, example: d3d8_jp.res, d3d8_en.res, d3d8_es.res, etc. If we do this we may also want to place these files under the sh2e folder.

[IlDucci]

What encoding are we attempting to use? Is there some reason we cannot use UTF-8 encoding? That should support all of the characters for all of the languages.

I'll do a quick explaining: as of right now, the game expects the strings in d3d8.res to be either in Shift-JIS (for Japanese characters) or in ASCII (for Western languages). The accented part of ASCII and Shift-JIS do not mesh at all. (Note as well that in order to get the Japanese properly read as Shift-JIS, we need to remove the "\h" part of the strings, and that also causes any ASCII text to output ingame as fullwidths).

Right now, to get the single-filed d3d8.res properly read in all languages, I have to "misencode" the file so it sticks to ASCII while also having Shift-JIS based characters for Japanese (basically I encode the Japanese into S-JIS, then open the file as ASCII and then paste the gibberish onto the working d3d8.res file). That is why I was suggesting splitting out the d3d8.res file into pieces.

I do not know if you can hijack the encoding the game expects and make it read, IDK, UTF-8 without any issues happening. I do know that while the Japanese encoding is custom-based, the order of the kanjis does match Shift-JIS'.

[elishacloud]

Could we save the file as UTF-8 on disk and then when the games starts up we load the file into memory and convert it from UTF-8 to whatever format the game requires? This way when the game needs the string we give it to the game in the format they need it.

For another option, I am ok if we store each of the languages in a different file, example: d3d8_jp.res, d3d8_en.res, d3d8_es.res, etc. This will allow some of the files to be in ASCII and others to be in Shift-JIS. But, as mentioned above, if we do this we may also want to place these files under the sh2e folder. It might be cleaner this way anyways.

[IlDucci]

If you can do the conversion, then that would be lovely. I am thinking under the supposition (probably of years where I was never, ever, being able to switch encodings in such a grand scale just like that) that it would be impossible to mod into the game without breaking either the Western languages or the Japanese sheet.

[elishacloud]

There are functions in Windows to do the conversion from UTF-8 to Shift-JIS and from UTF-8 to ASCII. Two examples here and here (see bottom of the thread).

If you want I can make a function where you can pass in the UTF-8 text and I can pass out the ASCII or Shift-JIS if you want.

However, I do like the idea of store each of the languages in a different file. This seems cleaner to me.

[elishacloud]

@IlDucci, I talked to @Polymega about this and it looks like we are planning to split the d3d8.res file up into multiple files and move them into the .\sh2e\etc\resources\ folder.

We are also planning to change the file name from d3d8 to something else, like maybe r_menu or something. If you have any ideas let us know.

Here is what I am thinking:

• r_menu_e.res
• r_menu_f.res
• r_menu_g.res
• r_menu_i.res
• r_menu_j.res
• r_menu_s.res

[IlDucci]

Sounds good to me.