farmerbb
Taoplan Arcade Shooters Extracting Tiles and Sprites from Zero Wing (and possibly others)
@stuckinstyx : Do you have plans to release scripts for the rest of the collection like Vimana or Frying Shark?
I'm working on it, but several of them have some interesting things going on. A bunch seem to have files that only use the bottom 4 bits of each byte, so I have to figure out how to massage them back into a format that FBNeo likes. Real life has been getting in the way of spending time on it for a little while. Families, jobs, and theatre performances really sap your energy sometimes.
Hi stuckinstyx, I have found the solution to extracting the tiles and sprites from Zero Wing in the GOG and Steam collections. Follow these directions:
After using binwalk (see https://github.com/farmerbb/RED-Project/issues/120) to extract the files look for 108F30 (I'm using the Steam release so it might be a different file in the GOG collection). It is 2,696kb in size. Open the file in a hex editor and copy the data from line 557,056 to 2,654,208 into a separate file named "sprites". It will be 2,048kb in size.
Now install Thonny at https://thonny.org/ for python on your computer and run the file "nibbleremovalandmerge" in the same directory as sprites. Just an FYI, a nibble is 4 bits or half a byte. This file will remove all of the extraneous 0s shown in the first nibble (high nibble?) of the byte. It will also merge the second nibbles (low nibbles?) together to form new bytes. It will output a file called "output" that will be exactly half the size (1,024kb).
3)Now put the bitwise.py program in the same directory as "output" and run the file. This will change the format of the sprites into another file called "bitwise".
4)Lastly use a program like BINMAN at https://jammarcade.net/binman-3-8-update/ to split the file "bitwise" in half. This will give you 2 512kb files. Deinterleave the files in the first half. This will give you 2 256kb files. Then deinterleave those files into 4 128kb files. Do the same for the second half. Deinterleave and deinterleave again. Eventually you will get these files:
o15-05.rom | 131072 | 4e5dd246 | 5366b4a6f3c900a4f57a6583b7399163a06f42d7 | tiles | 0 | good | no | | o15-06.rom | 131072 | c8c6d428 | 76ee5bcb8f10fe201fc5c32697beee3de9d8b751 | tiles | 1 | good | no | | o15-07.rom | 131072 | efc40e99 | a04fad4197a7fb4787cd9bebf43e1d9b02b2f61b | tiles | 40000 | good | no | | o15-08.rom | 131072 | 1b019eab | c9569ca85696825142acc5cde9ac829e82b1ca1b | tiles | 40001 | good | no | | o15-03.rom | 131072 | 7f245fd3 | efbcb3663d4accc4f8128a8fee5475bc109bc17a | sprites | 0 | good | no | | o15-04.rom | 131072 | 0b1a1289 | ce6c06342392d11952873e3b1d6aea8dc02a551c | sprites | 1 | good | no | | o15-01.rom | 131072 | 70570e43 | acc9baec71b0930cb2f193677e0663efa5d5551d | sprites | 40000 | good | no | | o15-02.rom | 131072 | 724b487f | 06af31520866eea69aebbd5d428f80e882289a15 | sprites | 40001 | good | no | |
I hope this helps. Ask any questions if this was unclear.
By the way I posted a new issue, because i was concerned it would get buried on page 2 of the thread. If there is a better way to do this, please let me know. Originally posted by @charlieglide in #186
Greetings, I have refined the process of extracting Zero Wing. If you have the Steam version, you should be fine. If you have the GOG version, make sure to rename the file that you extract with Binwalk that is 2,696kb in size to 108F30. To extract with Binwalk follow these directions: After you download the collection, open up the folder for the game. You will see an .exe file with the name of the game (e.g. "Zero Wing"). Click in the name of the window for the folder at the top of the page (e.g. where the folder says "Zero Wing"). Next hold the keys "alt+D" and type "cmd". Before we go any further, you will need to have BASH, SUDO and BINWALK installed on your pc. When you are in the command window type "bash". Then type "sudo binwalk". After you put in your password a list of binwalk commands will appear. For Zero Wing, type "sudo binwalk -e ZeroWing.exe --run-as=root". Binwalk will extract files to a folder entitled "ZeroWing.exe.extracted". The largest file with some data is 108F30 and is 2,696kb in size. If the file you have is not named this, please rename the equivalently sized file to this so that the python extraction will work. Run the python file I have attached (ZeroWingextract.py) in the same folder as 108F30 and run it.
It should extract the files into a zip folder named zerowing1 and zerowing
This file will work with Truxton. With binwalk use:
sudo binwalk -e Truxton.exe --run-as=root
Look for the extracted file that is 2,320KB file. If it is not named 108610 please rename it so that the attached file will work with it. Run the file attached and you will get a zipped Truxton file.
I've confirmed that you can do this with Outzone as well. I just don't have time to write the script tonight but will work on it soon. There are three files outzonec, outzoneh and outzoneb.
Greetings, I have refined the process of extracting Zero Wing. If you have the Steam version, you should be fine. If you have the GOG version, make sure to rename the file that you extract with Binwalk that is 2,696kb in size to 108F30. To extract with Binwalk follow these directions: After you download the collection, open up the folder for the game. You will see an .exe file with the name of the game (e.g. "Zero Wing"). Click in the name of the window for the folder at the top of the page (e.g. where the folder says "Zero Wing"). Next hold the keys "alt+D" and type "cmd". Before we go any further, you will need to have BASH, SUDO and BINWALK installed on your pc. When you are in the command window type "bash". Then type "sudo binwalk". After you put in your password a list of binwalk commands will appear. For Zero Wing, type "sudo binwalk -e ZeroWing.exe --run-as=root". Binwalk will extract files to a folder entitled "ZeroWing.exe.extracted". The largest file with some data is 108F30 and is 2,696kb in size. If the file you have is not named this, please rename the equivalently sized file to this so that the python extraction will work. Run the python file I have attached (ZeroWingextract.py) in the same folder as 108F30 and run it.
It should extract the files into a zip folder named zerowing1
Thank you for your great work, charlieglide. Two versions can be extracted from Zerowing.exe: Zerowing1 Set and Zerowing set. The program file for the Zerowing Set is at the top of the file extracted using Binwalk.
Zerowing Program files e697ecb9 o15-11ii.bin b29ee3ad o15-12ii.bin
Thanks for catching this Shigeshigeru. I will have to amend the ZeroWing script for the new files.
I have been trying to write a script for Outzone but I am still working on it. The file to work on is named 105E20 (for Steam--I don't know the name for GOG yet). In the file from 0-262,144 is a CPU file. Extract that file and deinterleave it. It is rom7.6h and rom8.6f. Put them in the folder named outzonec. Next, in 105E20 copy the data from 262,144 to 524,288 and then deinterleave this file into two files. named tp_018_07h.6h and tp_018_08h.6f. Please put these two files into a folder named outzoneh . Next, in 105E20 copy the data from 524,288 to 786,432 and then deinterleave this file into two files. The file names are tp07.6h and tp08.6f . Please copy these two files into a folder named outzoneb . Next, in 105E20 copy the data from 786,432 to 819,200 to a file. This is the first audio CPU. Please name it tp_018_09.3j and copy it to outzonec and outzoneh folders. Next, in 105E20 copy the data from 819,200 to 851,968. This is the second audio cpu and should be named tp09.3j and put in the folder named outzoneb . Next, copy the data from 851,968 to 1,900,544 to a file named sprites. Run the nibble removal and merge process on it (remove the 0 in the high nibble and merge all of the low nibbles). You will get a file that is half the size or 524,288kb. Now run the bitwise operation that we have been discussing in earlier posts (where you transform the 32bit blocks). After getting a file from the bitwise operation, please deinterleave this into two files. Then deinterleave those two files into four files. These should be named
tp-018_rom2.1c
tp-018_rom1.1e
tp-018_rom3.1d
tp-018_rom4.1b
All of these are 131,072 in size and should be placed in all three folders. Then in 105E20 copy the data starting at 1,900,544 to 2,949,120 (the end of the file) and split it into two files named tp-018_rom5.19h and tp-018_rom6.22h put them into all three folders. Lastly make two dummy files that are 32bytes in size and place them in all the folders. They should be named: tp018_10.rom10.18a and tp018_11.rom11.22c Older versions of MAME have different file names, so you may need to correct. As I mentioned, I will try to write a script when I get a chance.
I have attached the file for extracting Outzone for MAME. You will probably need to rename the files if you are using any older versions of the emulator. I tried using it on MAMEUIFX 175 and ARCADE 231, but I needed to rename the files. Here is the python code. Just follow the same procedure outlined above.
I have attached a file that will allow you to extract Fire Shark and its clones for MAME. I used the GOG version of Fire Shark. So when you extract it following the directions for Binwalk make sure that the file that is 3,616kb in your extracted folder is named 103600 Then just place the python file in the same folder and run it. The mame files are in the output folder.
I was able to successfully extract Slap Fight. It is the bootleg version slapfighb3. There were some files for alcon on there but I did not see any of the mcu yet. The proms for Slap Fight were there. I will have to post a script sometime in the near future. Just wanted to convey that it is possible to extract the bootleg version from the collection. Also, the Steam version of Zero Wing with file 108F30 has the zerowing (2p set) roms interleaved at 2,654,208-2,719,744 so as Shigeshigeru was saying I will need to add that to my original script.
Okay. I had a little trouble with this one but if you have Slap Fight by GOG you should be able to extract it in the same way mentioned above. Just use binwalk. Look for the file named A7450. If you have a different version like Steam, make sure it is the one that is 721kb in size and rename it. Now just put the file into the folder I have attached and go step by step. Run each of the steps in order Step1, Step2, Step3 and Step4. You will get two files with slapfighb3 and alcon (which has 2 files and the game will not run). slapfighb3 should work though.
slapfighb3-step-by-stepwithA7450.zip
Today I was able to extract Tiger Heli from the GOG collection. The playable rom in MAME is tigerhb1 (the first bootleg set) and tigerhb2 (the second bootleg set). I will post an extraction script in the near future.
Here is the extraction script for Tiger Heli from the GOG Collection. Run binwalk as described above. Then just extract the python files in the folders and put file a7430 that is 425kb in the folder with the 4 files. If you have the Steam version just look for the file that is 425kb and rename it to a7430 so that the script reads it correctly. Just go step by step. By step 4 you should have tigerhb1 and tigerhb2 files playable in MAME.
tigerhb1andtigerhb2extract.zip
I was able to extract "Hellfire" this evening. There are 3 sets including hellfire, hellfire1 and hellfire1a (older). I will post an extraction file in the near future.
I have attached the file to extract "Hellfire" from the GOG software. After you run binwalk on the Hellfire.exe, just put the file named hellfireextract.py in the folder with the 102480 file. If you are using the steam version whatever file is 2880kb needs to be renamed to 102480 for my script to work. It should extract the files into three folders named hellfire, hellfire1 and hellfire1a. I use slightly older versions of MAME so for MAMEUIFX175 no nag and for Arcade32 version 231, I had to rename the files to correspond to the old names. The zip files created here should work with the newest versions of mame, although the prom files are dummy files and the data does not match, the games will play fine.
Here is the extraction script for "Vimana". Same directions as above. Use binwalk. Then find the file that is in _Vimana.exe.extracted that is 3,360kb in size. In the GOG version that is 105180 so make sure you name the corresponding file to this name. Also make sure you have nibbleremovalandmerge.py and bitwise.py in the same directory as the file and vimanaextract.py when you run the latter. Also, when I ran this on MAMEUIFX175 NoNag (my favorite version) the sound rom was not emulated. I went up to Arcade231 and it worked fine, but I had to change seconds to run from 0 to 1 to get it to skip the game warning screen since the proms are dummy files. Also, you will probably need to rename the files if you are not using the newest version of MAME.
Here’s a partial extraction script I put together for Twin Cobra. There's also an incomplete set for Kyukyoku Tiger included, though unfortunately no sound ROM is present. I'm curious if the Twin Cobra sound ROM included in this set is shared between both games in the collection?
Anyway, it’s a bit of a disappointment — as others on the Milkchoco forums have warned, both Twin Cobra and Flying Shark are missing the DSP and MCU files. Bottom line: even with dummy files for the DSP and PROMs (which might not be necessary), the game won’t run — the DSP seems to be essential. When it starts there is a blue screen and it says RAM OK, ROM OK and SOUND OK and then it won't proceed any further.
For the Steam version, the file you're working with is FFB70 and it’s 1,696 KB in size. I haven’t checked the GOG version yet, so I can't confirm if it's different.
There may still be hope. RealRelativeEase has pointed to the Switch ports as a potential source for the missing files — see here: https://github.com/farmerbb/RED-Project/issues/223
Also, it’s possible the Evercade Toaplan Arcade 2 collection might contain the files we need. Has anyone confirmed whether dsp_22.bin and dsp_21.bin are present there? Could the bootleg MCU files for Flying Shark — 82s137-1.mcu through 82s137-8.mcu — be on those cartridges? And what about the official dsp_d70012u_gxc-02_mcu_71001 file?
Clearly, more research and analysis is needed. For now, I’m calling it a night — but here’s everything I was able to extract from Twin Cobra on Steam. Perhaps the Evercade cartridges and extraction technique as shown at https://milkchoco.info/archives/11224 is the key to getting these last two games ("Twin Cobra" and "Flying Shark"). Anyway, for what it's worth here is the script:
According to Shigeshigeru on this thread, Twin Cobra — and possibly Flying Shark — may be playable using the older "Shark" emulator. While this isn't an ideal solution, it might be the only viable option at the moment if you're looking to run the games outside the official software. There's more discussion on the site if you're interested.
Keep in mind that the software included in the Taoplan Evercade cartridges appears to be based on MAME 2016, so it's highly unlikely to include the official DSP dump.
Here is the link to the Shark emulator. I'm at work so I can't try it until I get home tonight: https://www.zophar.net/marcade/shark.html
I've attached a script that performs a partial extraction of Flying Shark, Flying Shark (bootleg), and related titles like Sky Shark and Hishouza (bootleg).
This was tested using the GOG version of the game. After running binwalk on flyingshark.exe, look inside the _flyingshark.exe.extracted folder for a file named FA920. That’s the file you'll want to work with. As long as your extracted file is exactly 1824KB, you should be good to go—just be sure to rename it to FA920 if it isn’t already.
Please note: these games won’t run out of the box. I haven’t created dummy files for the PROMs because I’m not sure they’re necessary—and to be honest, I didn’t feel like making them :). However, if you happen to have either:
The 8 MCU DSP files (each 1024KB, totaling 8192KB), or
The single 3072KB MCU file,
...and PROMs turn out not to be required, you may be able to get the game running, especially if you’re working with an original or bootleg arcade board.
Otherwise, our best bet lies in other sources—like the Evercade releases being discussed over at Milkchoco, or possibly the Switch ROMs shared by RealRelativeEase. Maybe something new will turn up down the line.
Anyway, I’m calling it a night—hope this helps someone out!
As I mentioned to Shigeshigeru a couple of weeks ago, I’ve revised the Zero Wing extraction script so that it now supports both zerowing1 and zerowing. I've attached the updated file here. Just follow the same directions outlined above for extraction.
I believe this just about wraps up the data mining from the Toaplan collections. Hope you all enjoy! P.S. I also uploaded this one above so that new downloaders get all of the files.
This is amazing work, @charlieglide. Thank you so much. Just picked up the Toaplan ultimate collection from the Steam summer sale, looking forward to trying these scripts out (and stuckinstyx's as well).
Thanks, farmerbb, for the kind words. If I discover any new methods for extracting additional compilations in the future, I’ll be sure to post them here on your site.
I especially want to thank ChatGPT for helping me recognize a key bitwise pattern that unlocked the sprites and tiles in many of these games. When it pointed out that each 32-bit chunk (or dword) in both the software and the target data contained the exact same number of 1s and 0s, I suspected a pattern was at play.
The process begins with a set of data from the GOG or Steam binwalk output file, where the top nibble is stripped out and the remaining lower nibbles are merged (as done in nibbleremovalandmerge.py). Once this is done, we get a new file which is half the size. Now a chunk or dword might, for example, look like:
A5 76 D3 4F
It can be broken down into binary bits using 1 for a set bit and 0 for an unset bit:
A = 1 0 1 0
5 = 0 1 0 1
7 = 0 1 1 1
6 = 0 1 1 0
D = 1 1 0 1
3 = 0 0 1 1
4 = 0 1 0 0
F = 1 1 1 1
From here, a bitwise transformation is performed by reading specific vertical bit columns:
-
Fourth column (upper right), read downward: 0 1 1 0 = 6
1 1 0 1 = D -
Second column, read downward: 0 1 1 1 = 7
1 0 1 1 = B -
Third column, read downward: 1 0 1 1 = B
0 1 0 1 = 5 -
First column, read downward: 1 0 0 0 = 8
1 0 0 1 = 9
So, applying the bitwise transformation to the dword A5 76 D3 4F gives:
6D 7B B5 89
This transformed output (of all of the dwords in the file), when double-deinterleaved, produces the correct sprite and tile data found in the original ROMs.
If any part of this explanation is unclear, I’m happy to clarify. But in essence, it acts like a cipher—one that could potentially be used to extract other compilations encoded in a similar way.
Have a Happy Fourth of July, everyone!
I cannot extract Truxton from my Steam copy.
Here is my truncated error:
line 125, in main
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'b65_12.7c'
The file that the ROMs are supposedly in is named 1099C0 and its CRC32 is 4C59EC13.
Notably, Truxton was the only game that failed to extract for me. All of the other games, which are also the Steam versions, extracted without any issue.
@DrAzathoth I ran into the same issue, I was able to fix it by modifying lines 110 and 111 in the Truxtonextract.py script to:
create_dummy_file("b65_12.7c")
create_dummy_file("b65_13.6d")
@DrAzathoth I ran into the same issue, I was able to fix it by modifying lines 110 and 111 in the
Truxtonextract.pyscript to:create_dummy_file("b65_12.7c") create_dummy_file("b65_13.6d")
Thank you for catching this error DrAzathoth and for fixing this Farmerbb. I replaced my original file with your corrected script in the earlier posts so that future users won't encounter the same issue. Thanks again to you both!
Some notes on Hellfire: I was not able to get hellfire1a.zip to load as-is using MAME (0.277). After renaming the file b90_03.11e to b90_03x.11e, the game loads, but there is no music (just sound effects). hellfire.zip and hellfire1.zip work 100% fine.
Also, for Tiger Heli, neither tigerhb1 nor tigerhb2 are working for me, as the sound ROM a47_03.12d isn't present in the extracted output.
Hi farmerbb, I have attached the corrected file for Tiger Heli. Thanks for catching the error. I've edited the post above as well.
tigerhb1andtigerhb2extract.zip
Here is the correction to the hellfire extraction script. I just double checked it. Thank you farmerbb for catching this. I will update the file above.
Thanks @charlieglide! Everything is working perfectly now with those fixed scripts 👍
Thanks @charlieglide , I finally had a chance to use your scripts and they work great. Even Flying Shark works if i use the mame2010 core in retroarch. So the only one missing for me is Twin Cobra.
I had given up figuring these ones out, so I really appreciate your hard work.
Hi stuckinstyx, I'm glad you found the scripts useful. The Evercade cartridges have some of the missing ROMs for the Toaplan games if you ever need them. Thanks for your scripts. I really appreciate it.