Implement Memory Manipulation on Windows

[Sparus42]

Using OCR to pull data seems like both a pain to setup and rather inefficient, plus it doesn't work on all data types one may want to use from a game. Implementing the ability to easily read from the program's memory fixes these issues, and depending on the game it may allow one to forgo capturing the screen entirely, avoiding the need for a CNN.

Writing to memory is less obviously useful, however it can assist in setting up the game into a certain state for training.

[Martyn0324]

I will copy the code to my files and test it out. Do these changes require a command in the GameAgent/Game Plugin code or are they simply upgrades for an automatic process happening behind the scenes?

Also, if possible, try writting a tutorial on how to use this feature(I don't know how to use Cheat Engine myself).

[Sparus42]

Yeah, this is a new feature that needs to be called in the game plugin or agent.

As for a tutorial, I can definitely write up something in the wiki, but Cheat Engine actually has a really good built-in tutorial in the form of some interactive puzzles that teach it a lot better than a text guide could (in the help menu, in case it doesn't show up on the first launch). Once you get whatever addresses you want it's simple enough, the game plugin I made using this should be a solid example to copy from.

[Martyn0324]

I've checked the CE's tutorial, but it doesn't seem to explain clearly how to get Hex adresses. I don't know if it's indeed not clear enough or if I'm just not paying enough attention, unless I already got the Hex adress and only need to add a 0x.

Also, I tested this feature on my pc with Jigoku Kisetsukan and the result was...pretty bad. The values seemed to vary between the correct value and a giant number. Try making this tutorial on how to get the arguments, perhaps I did something wrong.

Maybe a simple, direct tutorial page for the wiki on how to do that, in a similar way to what was done for extracting screen regions would be good.

If this feature is more efficient than OCR, it might be interesting. But the Cheat Engine part seems to narrow the probability of someone really using it (I tried using it with Bullet Heaven 2 and it didn't work, perhaps because the game got an online global ranking).

[Sparus42]

Yeah, I didn't exactly have any prior experience with memory manipulation myself, so it's not too bad to figure out depending on the game. Jigoku Kisetsukan just used memory pointers, which never came up in any of the games I tested so I didn't think to implement them before. It works perfectly now, though! I'll write up a guide that goes over everything in the next couple days, but for the moment you can try the code I used in the game plugin; should theoretically work for you too if we're on the same version.

def get_score(self):
        pointer_offsets = [0x0, 0x42C, 0xC, 0xBC, 0x8, 0x34, 0x10, 0x178, 0x0]

        score = self.window_controller.read_memory(0x003C943C, 8, pointer_offsets)
        try:
            score = struct.unpack('d', score)[0]
        except struct.error:
            score = None
        return score

(And unfortunately I don't have Bullet Heaven 2, so I can't test how well that one works. I doubt the online would be much of an issue, though, at least for the memory reading aspect.)

[Martyn0324]

Nice. Now, it indeed works fine.

Try reviewing the code while you write the tutorial, then I'll merge the pull request.

[Martyn0324]

I've decided to check how inefficient OCR could be, as you mentioned, and used datetime to see the delay between each action in Bullet Heaven 2.

The only difference between the codes used in the videos is the OCR. I only used OCR to get the score. Lifes and Bombs were gotten using SpriteLocator (Even so, it seems that both didn't work as expected).

Also reminder that I'm using a GeForce GTX 1650 from my personal computer.

PS: Don't mind the bugged cursor out of the gaming screen. It's from the recorder app.

With OCR:

https://user-images.githubusercontent.com/28028007/142750021-3e5a234d-d020-40b5-9893-33ab3e8fd9ed.mp4

Without OCR:

https://user-images.githubusercontent.com/28028007/142750084-72a62598-2994-46ea-a7fc-49e187ea1a09.mp4

Pity that some games don't allow Cheat Engine. Maybe we could try to work with Cheat Engine and, when necessary, try to figure out some alternate means to get values.

[Sparus42]

Interesting! I got the guide written up here, didn't add it to the main page yet though. Take a look and let me know if I should clarify anything else. https://github.com/Martyn0324/SerpentAI/wiki/Memory-Manipulation-on-Windows

Also, I don't think there are many games, if there are any at all, that memory reading just straight won't work on. I downloaded Bullet Heaven 2 to figure out what the issue was, and it turns out that the game doesn't actually keep the combined score in memory. You'll just have to add and multiply all the separate score values it shows on the end screen yourself, which should be simple enough. I only tested a few other values, but they all worked fine too.

[Martyn0324]

Nice. The wiki is ok, but I think it's better to improve it a bit, since memory manipulation seems quite complicated, at least when pointers are involved.

First, the 3rd image in the section "Finding the Pointers" isn't appearing, maybe something went wrong, this probably got me confused(I'm following the tutorial while trying to get values for Jigoku Kisetsukan). I got an address using pointers but, in the end, it was just another pointer and the value became "???" when I restarted the game.

Try reviewing and editting the wiki supposing that the person who will read is an idiot. This might help a lot.

Some games also don't allow for the execution of Cheat Engine - CE seems to be in a blacklist, somehow - so some apps can be useful as alternatives to do memory manipulation. Unfortunately, though, the best one, ArtMoney, isn't open sourced.

[Martyn0324]

I've decided to check how inefficient OCR could be, as you mentioned, and used datetime to see the delay between each action in Bullet Heaven 2.

The only difference between the codes used in the videos is the OCR. I only used OCR to get the score. Lifes and Bombs were gotten using SpriteLocator (Even so, it seems that both didn't work as expected).

Also reminder that I'm using a GeForce GTX 1650 from my personal computer.

PS: Don't mind the bugged cursor out of the gaming screen. It's from the recorder app.

With OCR:

lerdo.mp4 Without OCR:

rapido.mp4 Pity that some games don't allow Cheat Engine. Maybe we could try to work with Cheat Engine and, when necessary, try to figure out some alternate means to get values.

Disregard this

In the "Without OCR" code, I actually forgot to include my save function, which is:

if self.agent_actions.current_step % 100 == 0:
            self.agent_actions.save_model()
            self.agent_mouse.save_model()

And THIS is what really causes the delay between actions. The OCR indeed causes a delay, but what makes it jump from 2 seconds to 25 seconds is this operation. The thing is...I'm not using self.agent_actions.current_step because of this issue, so the result is that my model is being saved after each step.

Still, maybe the memory manipulation can be useful when dealing with more complex games that would require too much OCR...or when OCR doesn't help at all(like in Jigoku's case, where the OCR usually makes too many mistakes).

[m1ndgames]

Hey guys, im late to the party, but i'd like to contribute:

I also found out that ocr is slow, and in my usecase it wasn viable, so i grabbed the health values from the memory, you can find a snippet here:

https://github.com/m1ndgames/SerpentRoboGameAgentPlugin/blob/master/files/helpers/memory.py