Original game mechanics and Wiki discussion

[Allofich]

I've reversed the game's formula for spell points. Of course, the results (the spell point multipliers per class, etc.) are already known information, so I guess this is mainly just for curiosity or for emulating the original game and using the .exe data.

I don't know whether this is also used for NPCs or if it's just the player.

uint GetMaxSpellPoints(NPCDATA* npcData, uint currentIntelligence)
{
    // See the wiki page on "Save File Formats" for the NPCDATA structure
    byte class = npcData->class; // Get the character class

    // Default of 0 spell points. (For non-spellcasters)
    uint spellPoints = 0;

    // If class is a spellcaster and not a sorcerer
    if ((class & SPELLCASTER_MASK != 0) && (class != 0x23))
    {
        spellPoints = currentIntelligence; // 1x intelligence spell points (Bard)
 
        // If not a bard
        if (class != 0xe6)
        {
            uint spellPointBonus = currentIntelligence; // 2x intelligence spell points (Mage, Healer)

            // For the remaining classes, a byte array in the executable file is used.
            // The array has 6 values, ordered by class ID.
            // If the value gotten is 2, then the total will stay as the above (2x intelligence).
            // If the value gotten is 0, then the total will become 1.5x intelligence.
            // For any other value, the total will become 1.75x intelligence.
            if (SpellPointModifiers[class & ID_MASK] != 2)
            {
                if (SpellPointModifiers[class & ID_MASK] != 0)
                {
                    spellPoints = currentIntelligence + (currentIntelligence >> 2); // Add 0.25x intelligence
                }
                spellPointBonus = currentIntelligence >> 1; // Add 0.5x intelligence
            }

            // Calculate the total
            spellPoints = spellPoints + spellPointBonus;
        }
    }

    // If class is a sorcerer
    if (class == 0x23)
    {
        spellPoints = currentIntelligence * 3;
    }
    
    return spellPoints;
}

The "SpellPointModifiers" array location for aExeStrings.txt is 0x40110. For acdExeStrings.txt it is 0x40750. It looks like:

SpellPointModifiers
      02 Mage
      00 Spellsword
      01 Battlemage // "1" here, but any value other than 2 or 0 has the same effect
      FF Sorcerer // Unused
      02 Healer
      00 Nightblade
      FF Bard // Unused

[afritz1]

This is great. I can make you a collaborator so you can add this and anything else you like to the wiki.

[Allofich]

Thanks. I've never edited a wiki before but I'll give it a shot later if you make me a collaborator. There's a few other things I've also found out that I can also add.

[Allofich]

@afritz1, I've added a new page with the above.

I thought about where to put the spell points formula. I considered the existing pages but ended up making a new one called "Player and NPC stats".

I was thinking it may be good to move the NPCStats information from "Save File Formats" into "Player and NPC stats", and then just refer to NPCStats by name (maybe a URL link if that is possible) in the "Save File Formats" page, but for now I won't touch any of the existing text. @Carmina16, I welcome your input if you would like to say anything.

Also, things I add to the wiki will probably be a little bit inconsistent with the way Carmina16 wrote, in that I want to write pseudocode in C. Carmina16 put the offset locations of data as offsets from the start of the decompressed A.EXE (1.06) to the start of the data, whereas I was thinking of putting (as I did above) the offsets as they need to be entered in the aExeStrings.txt and acdExeStrings.txt files. For aExeStrings.txt that seems to mean adding 0x3D30 to the "from-start-of-file" value, while for acdExeStrings.txt that seems to mean adding 0x3F50 to the "from-start-of-file" value for decompressed ACD.EXE. (aftritz1, please correct me if I'm wrong about these values)

As for this issue, for now I think I will keep it open as a place for discussion about original game mechanics and the wiki, if that is all right with you afritz1.

[Allofich]

Hmm, well it would be a pain to go back through all of the existing Wiki pages and make the offsets refer to the aExeStrings.txt and acdExeStrings.txt values, so I think i will just put the "from-start-of-file for decompressed A.EXE" value as Carmina16 was doing.

[Allofich]

Changed the page to "Player stats" since NPC-specific stuff is already going in the "NPC" page.

[Allofich]

Added information on spell and regeneration assignment to the NPC page.

[Allofich]

Added information on the length of time that top-of-screen messages are shown to a new page called "Timing".

[Allofich]

@afritz1, do you receive notification when I add to the Wiki? If so, I won't bother mentioning here when I've added something.

[afritz1]

I get notices in my activity feed which is great. Thanks for adding all that stuff. I'm working on location refactoring right now but I will try getting to the things you added once I get that far.

The reason why the aExeStrings.txt offsets are different from Carmina16's is because the PKLITE decompression code I implemented is missing a chunk of data -- I think it's the original game's executable code. It wasn't needed since we only need to decompress the original executables for the global constant data.

[Allofich]

I added lockpicking information, but unfortunately I don't know where the lockpicking difficulty of a door comes from yet. It's easiest to reverse the "end tips" of things that are closest to output, like code just before string messages that indicate success or failure. If you or Carmina16 know where door lockpicking difficulty comes from then you could probably get lockpicking running pretty quickly.

I get notices in my activity feed which is great.

OK, I won't post here just to say when I added something anymore, then.

[afritz1]

I already have some code for lock difficulty. It comes from the level data in .MIF files. https://github.com/afritz1/OpenTESArena/blob/075ce256ccb4d446a7d1ebd90f7d8a2e33a12fb5/OpenTESArena/src/Assets/ArenaTypes.h#L36 https://github.com/afritz1/OpenTESArena/blob/94577cc802aa8a730aae187fd9dea2e83205e0ea/OpenTESArena/src/Assets/MIFFile.cpp#L313 https://github.com/afritz1/OpenTESArena/blob/9b885e1acda8da8c0a89bc8c2975e233f8873967/OpenTESArena/src/World/LevelData.cpp#L1056

[afritz1]

You can add syntax highlighting to code blocks if you put the language right after the three back ticks. Like:

```C++

[Allofich]

You can add syntax highlighting to code blocks if you put the language right after the three back ticks.

Thanks, I updated the pages and changed the data types like "byte" and "uint" to be primitive types like "unsigned char" and "unsigned int" so they will be highlighted as data types.

[afritz1]

Some unsolved problems I have on my mind that I just want to write down and maybe looking in the original executable would help. I didn't see them in the wiki anywhere but maybe I missed them:

Interior raised platform texture coordinates (#129)

• https://github.com/afritz1/OpenTESArena/blob/2d2bf2eb04aef9ad8fd046ba0c75b19e729e1a37/OpenTESArena/src/Assets/ExeData.h#L502

Wilderness raised platform sizes and heights (similar to #129)

Moon 1 and 2 positions (uses strange integer coordinates but I couldn't get it to look right yet. Ideally we could figure out what those numbers mean in regular Vector3 values)

• https://github.com/afritz1/OpenTESArena/blob/a8de5423a47b71f56eedf902757e1b55c8518420/OpenTESArena/src/Rendering/SoftwareRenderer.cpp#L1633

[Carmina16]

Those various arrays are @4bf36.

[afritz1]

Sorry, I meant that I have the arrays but I don't know how to interpret them for all cases -- for interior raised platforms and wilderness raised platforms. They seem to be interpreted differently. Here is where they are used currently: https://github.com/afritz1/OpenTESArena/blob/2b0336f868a02f0c970eb4dd4bcf79ddeb93c7f2/OpenTESArena/src/World/LevelData.cpp#L690

[Allofich]

I know that issue is for an interior, not the wilderness, and so this may not be useful, but does this snip from the original look to give the same results as what your code is doing?

Decompiled original in pseudocode:

if (worldType == WorldType::Wilderness)
{
   for (int i = 0; i < 56; i++)  // 56 entries covering Box 1A, 1B, 1C, 2A, 2B.
   {
     int value = BoxArrays[i] * 192;
     BoxArrays[i] = value / 256 | (value / 512) * 256;
   }
}

Your code: https://github.com/afritz1/OpenTESArena/blob/acbb82d06fe374f8a632535572bb12bae38cfae4/OpenTESArena/src/World/LevelData.cpp#L704

[afritz1]

Looks useful but I don't immediately know how to work with BoxArrays. Is int 16 or 32 bits in your example? I'll look at it more when I have time.

[Allofich]

By BoxArrays I mean the combination of the arrays of Box 1A, 1B, 1C, 2A and 2B, treating them as if they were all one array. They are all contiguous in the .EXE file.

The calculations done are 16-bit, so I guess that shouldn't be an int. I'll change it to an unsigned short.

Edit: The final value in Box1BWallHeightTable, 190h(400), will overflow beyond the 16-bit registers being used when multiplied by 192, so I wonder if it looks correct in the original game?

[afritz1]

Thanks, will look at it after this current attempt with city entrance jingles.

[Allofich]

Wait, sorry, it should be 32-bit after all. Changed it back to int.

[Allofich]

Another similar bit of code exists dealing with the BoxArrays. I think while the above matches your !boxScale.has_value() case (using 192), this one matches the boxScale.has_value() case. But, it seems like while your code always uses a 32 here, maybe the original code can use anything over 31.

  int a = funcCall() // This gets some value, it may be like your "inf.getCeiling().boxScale"
  if (a < 33)
    a = 100; // Don't know what this is
  int b = -a; // Or this
  
  int c = funcCall() // Same function as above. Gets the next value from the file?
  if (c > 31) // I guess this value is like the "32" you use?
  {
    for (int i = 0; i < 56; i++)  // 56 entries covering Box 1A, 1B, 1C, 2A, 2B.
     {
     int value = BoxArrays[i] * c;
     BoxArrays[i] = value / 256 | (value / 512) * 256;
     }
  }

With your familiarity with the code, I'm hoping maybe some of these values will be recognizable to you. Or maybe Carmina16 knows more.

[Allofich]

There is one other bit of code I can see regarding the BoxTables, which might be relevant to https://github.com/afritz1/OpenTESArena/issues/129. Like the above snippets, I don't know what this data represents, but maybe with your knowledge you can recognize things and piece it together.

if (*(unsigned char *)data != *(unsigned char *)(data + 1)) // If the next byte of some data (.INF or .MIF file?) being read differs from the current
{
  int i = (*(unsigned int *)(data + 1) & 7); // Height index?
  if (Player is outside) // Outside in city or in wilderness
  {
    if (Player is outside in city)
      i += 8; // Use BoxB
    else if (Player is outside in wilderness)
      i += 16; // Use BoxC
  }
  short global1 = Box3AWallHeightTable[i] & 0x3f;
  short global2 = Box1AWallHeightTable[i];
  short global3 = -global2;
  int index2 = *(unsigned int *)(data + 1) >> 3 & 0xf;
  short global4 = Box4WallHeightTable[index2];
  short global5 = 64 - (global4 + global1);
  if (PlayerInWilderness)
        index2 += 16; // Use BoxB

  short global6 = Box2AWallHeightTable[index2] + global2;
  short global7 = -global6;
  short global8 = Unknown[(byte *)(data >> 4 & 0xf)]; // Unknown[] is an empty-at-start array. 
  unsigned char global9 = 1;
  unsigned char global10 = 2;
  short global11 = global7;
  short global12 = global3;
}

The above pseudocode allows spilling over from one array to another. So index2 += 16 will cause Box2AWallHeightTable[index2], which is 16 elements, to actually read from Box2BWallHeightTable.

[Carmina16]

The layout seems to be:

WORD dungeon_start[8];
WORD city_start[8];
WORD wilderness_start[8];
WORD interior_thickness[16];
WORD wilderness_thickness[16];
WORD source_copy[56]; // unscaled values of those above
WORD unknown1[8];
WORD unknown2[8];
BYTE unknown3[16];

There are only 2 unknown1 arrays, but the game addresses them as if there were 3, in the same order as the first 3 arrays.

The values from unknown1/2 and unknown3 are used as follows:

A <- unknownX[platform_start_id] mod 64
B <- 64-unknown3[platform_thik_id]-A

Those seems to be used in texture mapping of some sort. unknown3[...] is probably the height of 64x64 texture in texels, and B is the starting row of the texture.

[ratmonger]

Hello! I have stumbled into here quite late. But I am quite hopeful that I could help or learn a thing or too. I do not intend any actual coding work on this project, but I was hoping to view the assembly and reverse engineer the game. Provide any details about game mechanics and operations. I currently was intrigued how the magic defense system functions. Anyway... I recently installed "sourcerer" which is a decompiler for 16 bit programs that was used back in the day. Using that I was able to export dissasembled assembly code. But it is quite much for me. It is over 1000 lines of code and I have little idea where to start. I am only a college student and have had a little bit of progress with assembly using linux and various c programs. The dissasembled code lacks any functional call names or anything that can make it easy to read. For an idea of my assembly experience, I have worked with the well known "bomblab.c" that you can debug in GDB on linux. I'm in a course that covers assembly at the moment. I'm also quite eager to help though, I just have no idea where to start.

I saw people previously mentioned a decompressed A.EXE and I have found a website which seemed to reference strings they observed in the executable. I would also like to see this decompressed version. If anyone were to post a guide how to readily dissasemble the executable into something that proves easy to read along with any work, I think the collaborative effort would go much smoother and help the project move along!

[afritz1]

I haven't invested time in disassemblers/tools like Ghidra, but if you want to play around with the decompressed A.EXE/ACD.EXE used by OpenTESArena, start with ExeUnpacker.cpp and try writing this->exeData out to a file. The VFS::Manager code could probably be replaced with a simple fopen() or something. https://github.com/afritz1/OpenTESArena/blob/214998a15232b8535be0c72a69ffa6b676344c02/OpenTESArena/src/Assets/ExeUnpacker.cpp#L208

The project's HexPrinter utility makes it easier to read. https://github.com/afritz1/OpenTESArena/blob/214998a15232b8535be0c72a69ffa6b676344c02/components/utilities/HexPrinter.h#L9

I did a bomb lab too, lost a point cause I was a noob with breakpoints ;)

[ratmonger]

Thanks! (once with time again i wil delve into this, i may postpone work on this til winterbreak) I would hope Allofich might share how he derived some of the code formula for some of the game mechanics.

Also here is dissassembled assembly code of the unpacked ACD.EXE via sourcerer. ACD1-export.txt

[Carmina16]

I recently installed "sourcerer" which is a decompiler for 16 bit programs that was used back in the day.

You'll need an interactive disassembler like IDA or ghidra, and the debug version of Dosbox. Luckily, Arena is a real-mode application, so it would be easy to debug.

I would start with investigating strings and functions that use them, then identifying the game logic and data structures. You will also need the documentation on DOS, VGA and LIM/EMS interrupts and structures.