Controlling FCEUX externally (from another program)

[kulbhushanchand]

Thanks for the awesome emulator...

I am trying to control FCEUX externally (preferably from MATLAB), and the closest thing I was able to find is sending keystrokes to the emulator (for loading ROM, changing game speed etc.) I searched the documentation and not able to find anywhere regarding my doubts -

1. Does FCEUX implements any COM interface via which I can control it through another program
2. Does FCEUS have any activeX control interface

I have posted a question on stackoverflow also - https://stackoverflow.com/questions/53664513/sending-control-signals-to-fceux-emulator-from-matlab

[vadosnaprimer]

@spiiin do you happen to have anything in that department that could be shared?

[spiiin]

I have PoC of controlling fceux via Jupyter Notebook (interactive Python) through host lua-script https://www.youtube.com/watch?v=c3D5gljbkO0

Fceux has built low-level tcp-socket support, so it can read/write data from it. I created quick and dirty RPC json-protocol for receive commands via sockets: https://gist.github.com/spiiin/6ffe87ba37701acafca1f5579b1ffe14 (it's just a sample, how things may work, not ready solution).

It can be enough for many tasks - creating AI/bots, debugging games, creating 3rd party tools, injecting code and data in games. It's limited by several things - PPU debugging and callbacks that can be called thousands of times per frame.

Another solution is to use emulators with injected callbacks system - https://nintaco.com/api.html

[kulbhushanchand]

@spiiin Thanks for the directions and gist.

I wanted to use change the in-game speed on the fly, and the easiest solution so far is to send the keystroke from MATLAB. (in FCEUX there is option). I am intentionally avoiding python, since my server is in MATLAB (using UDP/OSC protocol)

1. As you have shared, lua listener looks promising, but is there any documentation for the beginners like me?

2. Currently I am able to send a stream output in MATLAB over UDP/OSC (Open sound Control) protocol. Is there any generalized lua listener, which makes FCEUX a slave and able to accept all the incoming commands over UDP/OSC ?

I am preferring UDP over TCP/IP, since in the former there is no need for connection to remain established between server and receiver, so it can tolerate packet drops.

[kulbhushanchand]

I have PoC of controlling fceux via Jupyter Notebook (interactive Python) through host lua-script https://www.youtube.com/watch?v=c3D5gljbkO0

This video is very much what I want to do, but using MATLAB not python. Any idea how can I port it over to MATLAB? Does the listener.lua remains same?

[spiiin]

You must write Matlab code to connect to the socket and send commands via it yourself. Lua code can be the same, it is just listening commands from the socket and execute it. Any application can connect and send commands to the emulator. What type of documentation do you want? http://www.fceux.com/web/help/fceux.html?LuaScripting.html - there are all functions available from fceux, nothing other not needed.

[kulbhushanchand]

Hi @spiiin , Thanks, I managed to get the script working with MATLAB and its working well. (Python in Matlab works a bit differently) I am following the same link and your youtube video.

A little doubt - I tried to change the onscreen coin values of Super Mario Bros Game, by editing the hex values -

memory.writebyte(hex2dec('07ED'),hex2dec('5'))
memory.writebyte(hex2dec('07EE'),hex2dec('3'))

The problem is on-display values doesn't immediately changes, but only when mario gets another coin. Do I have to change some other memory location for force screen update.

I am following the Mario Ram map from here - http://www.thealmightyguru.com/Games/Hacking/Wiki/index.php/Super_Mario_Bros.

[spiiin]

Your question related with the game code itself, not the emulator. I didn't explore SMB code. Coins are drawn as background object, and I didn't know, when and how background updated in this game. You may need to update not only ram (logic description of coins) but also ppu memory (tiles on screen).

[kulbhushanchand]

Sorry for not making myself clear, I meant to say the number of coins I have

I was able to change the RAM values to 53 (for example)

memory.writebyte(hex2dec('07ED'),hex2dec('5'))
memory.writebyte(hex2dec('07EE'),hex2dec('3'))

and changed value is shown on screen only when I acquire a new coin. and till them it is shown as 02 (or whatever the previous value was). I want the screen value to update immediately following the above update in RAM values

[spiiin]

Number of coins also drawn at background, ppu addresses 206D-206E

[kulbhushanchand]

That means when mario takes a coin, the game logic redraws the coin values.

Which function to use to force refresh/redraw the background?

[spiiin]

Rendering is controlled by the game code, there is no function in emulator or lua for this

[kulbhushanchand]

Looks like PPU memory access in lua is also not possible. ref - https://stackoverflow.com/questions/41954718/how-to-get-ppu-memory-from-fceux-in-lua

I even tried to change the score values by updating RAM, and likewise in above problem, the score GUI text updated only when mario do an action which results in increment in score.

I'm looking for a workaround 🤔

[kulbhushanchand]

@spiiin Can you have a look on the second solution to this SO question (below), where someone points out that PPU memory can be assessed with new PPU core option in FCEUX

https://stackoverflow.com/questions/41954718/how-to-get-ppu-memory-from-fceux-in-lua

[spiiin]

Ah, I forgot, fceux lua-script really has poor support of debugging and changing ppu state. The best solution is to implement needed function yourself as I do for ppu.readbyte/readbyterange functions :)

Other solutions are:

• use functions from second solution from stackoverflow, it must worked
• find needed function in SMB code and call this function from lua code (it also need to implement own lua function, that will implement call - change cpu registers and stack pointer)
• just draw your own numbers over the top of emulator picture: gui.rect for fill solid color, and gui.text for render text string.

[kulbhushanchand]

Thanks for the directions.

I may go for easy solution i.e. displaying overlay using gui function, but will do check the stackoverflow solution. Calling in game logic for gui update may be the lengthy approach.

I also commented on your GistPage - https://gist.github.com/spiiin/6ffe87ba37701acafca1f5579b1ffe14

--["gui.gdscreenshot"] = gui.gdscreenshot, --[gui.gdoverlay] = gui.gdoverlay,

You have commented out some functions. Was there any specific reason?

[spiiin]

https://gist.github.com/spiiin/6ffe87ba37701acafca1f5579b1ffe14#gistcomment-2789503

[kulbhushanchand]

Matlab returns error with the functions where string is returned by lua. (For returns of type int it's working fine). Any suggestions?

In Matlab -

In Python - (same problem in python too)

[spiiin]

It's a bug inside json library - it packed boolean values wrong. I fixed it in commit: https://github.com/spiiin/json.lua/commit/e792cd0336219d71161a805bbde2889ea6883d0a (Author of original json.lua didn't accept it, because in pure lua all works fine, but inside fceux some 3rd party library redefine tostring function somehow). So, you must take my fork of json.lua: https://github.com/spiiin/json.lua

[kulbhushanchand]

Thanks, working fine.

[kulbhushanchand]

Shouldn't the function emu.getscreenpixel returns all the rgb values of a pixel ? I received only an integer -

[spiiin]

Yes, I missed, that function returns 3 values instead of 32bit packed integer.

[kulbhushanchand]

Any plans to develop this project ? You may start a new repository to which others may contribute. It looks like presently this gist is the only available solution exists for communicating with FCEUX (or maybe I'm not searching in the right way)

[vadosnaprimer]

Agreed, every once in a while someone needs some headless variation of some emulator, and since this is basically one, it'd be very helpful to have it up.

[kulbhushanchand]

@spiiin

Yes, I missed, that function returns 3 values instead of 32bit packed integer.

Any plan on adding this missed function? Unfortunately my lua skills are not advanced enough.

[spiiin]

https://github.com/spiiin/fceux_luaserver Maybe I'll add some docs, fixes, and features when I have time to use it myself.