Fix asset conversion for HD + Rise of the Rajas (Version 5.x)

[FolkertVanVerseveld]

It is not possible to run OpenAge with current Steam version of Age of Empires HD + Rise of the Rajas (Version 5.x). The run script fails to convert the assets saying:

WARN [py] You have installed incompatible game version(s):
WARN [py]  * Age of Empires 2: HD + Rise of the Rajas (Version 5.x)

The complete run log:

INFO [py] launching openage v0.3.0-857-gc9419b8
INFO [py] compiled by GNU 5.4.0
INFO [py] running in DEVMODE
INFO [py] No converted assets have been found
INFO [py] Game specification version file not found.
INFO [py] Converting graphics, interface, metadata, sounds
INFO [py] Will save to 'b'/home/dev/openage/assets''
Could not find any installation directory automatically.
Please enter an AOE2 install path manually.
> /media/dev/ISOIMAGE
WARN [py] You have installed incompatible game version(s):
WARN [py]  * Age of Empires 2: HD + Rise of the Rajas (Version 5.x)
WARN [py] You need at least one of:
WARN [py]  * Age of Empires 2: The Conquerors, Patch 1.0c
WARN [py]  * Age of Empires 2: Forgotten Empires
WARN [py]  * Age of Empires 2: HD Edition (Version 3.0+)
WARN [py]  * Age of Empires 2: HD + Forgotten Empires (Version 4.0+)
WARN [py]  * Age of Empires 2: HD + African Kingdoms (Version 4.7+)
 ERR [py] game asset conversion failed
converting from '/media/dev/ISOIMAGE'

The provided image path is the Steam version (2017-03-31) of HD + Rise of the Rajas + The African Kingdom + The Forgotten Empires.

[TheJJ]

This is rather tricky as the asset format changed quite a lot. Mainly #720 and #141 have to be implemented for this. The UI piece generation also has to be updated.

[FolkertVanVerseveld]

I don't think I can really help resolve these asset related issues, because I currently only own the Conquerors and Age of Kings for macOS 9 (which is unsupported of course) and the steam Age of Empires bundle (HD + Rise of Rajas + The African Kingdom + Forgotten Empires).

I could try to work on some other issues and come back to this one once I can get the project up and running. Or maybe I could try to also add support for Conquerors and Age of Kings for macOS 9?

[TheJJ]

Sure, the macOS version will likely only have a different engine binary, the data files should be the same so detection and conversion should be no problem. The language.dll might be funny though.

[samuelgrigolato]

I'm taking a look at the conversion procedure, and was able to somewhat get things in the right direction (i.e. there seems to be 2 new hit_classes in the new asset version that weren't present in the old ones), but quickly noticed that it will be hard to reach anywhere close to finish this without some kind of documentation for the new asset descriptor file formats. How do you go about this?

I mean, how can I figure out the differences between the old and new descriptors, so I can update the Exportable types accordingly? Is there anything to aid me in this procedure or is it just plain guessing at this point? Maybe a generic viewer for the DAT file or something.

[heinezen]

genieutils should be a good source for the data conversion.

[TheJJ]

@samuelgrigolato the differences should be implemented in genieutils. the best dat viewer is AGE, which uses genieutils.

[FolkertVanVerseveld]

Wow, it has been a long time since I first requested this. I've also noticed some additional problems during asset conversion:

• SLP conversion fails because some commands are unimplemented. IIRC lower nibble == 5 is unimplemented which is used for terrain tiles and buildings.
• Menu backgrounds use a custom format.

The default palette is also id 50500, but most menu backgrounds use a different palette, but I've figured out most of the background file format for AoE and RoR.

If I get my environment up and running again I will create some patches to fix the asset conversion.

@TheJJ, what things are still relevant or most important to implement at the moment? Would you mind giving my a few starting points?

[TheJJ]

Awesome! Related to asset stuff, the most important part is the creation of asset packs. In those, nyan files and media files are stored in a structured way. @mic-e (general pack structure, media files conversion) and @heinezen (nyan structure from the dat file) are designing those, but of course you can join and contribute. We want to do things step by step, so we have to design the pack format/content in an extensible way to prevent having to implement everything at once.

For more interactive discussions I'd suggest you to join our matrix/irc channel :)

[heinezen]

Fixed by rewriting everything in the converter and engine :tada: