Emulator fails to emulate 8051 code, raising an exception

[cyrozap]

Describe the bug

When I try to use the emulator tool on an 8051 binary, it fails, raising the following exception:

Offset must be between 0x0 and 0xff, got 0x1000 instead!
ghidra.program.model.address.AddressOutOfBoundsException: Offset must be between 0x0 and 0xff, got 0x1000 instead!
	at ghidra.program.model.address.AbstractAddressSpace.makeValidOffset(AbstractAddressSpace.java:620)
	at ghidra.program.model.address.GenericAddressSpace.makeValidOffset(GenericAddressSpace.java:21)
	at ghidra.program.model.address.GenericAddress.<init>(GenericAddress.java:55)
	at ghidra.program.model.address.GenericAddressSpace.getAddress(GenericAddressSpace.java:88)
	at ghidra.app.plugin.core.debug.service.emulation.ProgramEmulationUtils.allocateStack(ProgramEmulationUtils.java:300)
	at ghidra.app.plugin.core.debug.service.emulation.ProgramEmulationUtils.doLaunchEmulationThread(ProgramEmulationUtils.java:376)
	at ghidra.app.plugin.core.debug.service.emulation.ProgramEmulationUtils.launchEmulationTrace(ProgramEmulationUtils.java:347)
	at ghidra.app.plugin.core.debug.service.emulation.DebuggerEmulationServicePlugin.emulateProgramActivated(DebuggerEmulationServicePlugin.java:455)
	at docking.action.builder.ActionBuilder$1.actionPerformed(ActionBuilder.java:48)
	at docking.DockingActionProxy.actionPerformed(DockingActionProxy.java:47)
	at docking.menu.ToolBarItemManager.lambda$actionPerformed$0(ToolBarItemManager.java:129)
	at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:318)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:771)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:722)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:716)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
	at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:741)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)

---------------------------------------------------
Build Date: 2023-May-12 1514 CEST
Ghidra Version: 10.3
Java Home: /usr/lib/jvm/java-17-openjdk
JVM Version: N/A 17.0.7
OS: Linux 6.3.1-arch2-1 amd64

To Reproduce

1. Open an 8051 program in the emulator.
2. In the "Debugger" menu, click "Emulate Program in new Trace".
3. Exception is raised.

Expected behavior

The emulator does not raise an exception.

Environment (please complete the following information):

• OS: Arch Linux
• Java Version: 17.0.7
• Ghidra Version: 10.3
• Ghidra Origin: Arch Linux

[cyrozap]

I did some investigation into the issue, and it seems there are two problems:

• The stack base is hard-coded to 0x1000: https://github.com/NationalSecurityAgency/ghidra/blob/4b99900d2f2ce6efc444aed979f6c17b1f49a988/Ghidra/Debug/Debugger/src/main/java/ghidra/app/plugin/core/debug/service/emulation/ProgramEmulationUtils.java#L300
• The stack size is hard-coded to 0x4000: https://github.com/NationalSecurityAgency/ghidra/blob/4b99900d2f2ce6efc444aed979f6c17b1f49a988/Ghidra/Debug/Debugger/src/main/java/ghidra/app/plugin/core/debug/service/emulation/ProgramEmulationUtils.java#L376

The 8051 only supports a stack of (at most) 256 bytes, but in practice the stack is usually smaller to make room for global variables in INTMEM space. At reset, the stack pointer SP is set to 0x07 (so the first value pushed to the stack is written to INTMEM:0x08), but most firmware I've looked at will initialize SP somewhere between 0x3F and 0x7F, inclusive.

I know the 8051 is a weird architecture, so a fix for this issue might only reveal more problems with 8051 emulation, but it would be nice for Ghidra to support it. Existing emulators are designed more for simulating whole firmware images as debugging aides, and not for emulating individual functions or as reverse engineering tools.

[d-millar]

@cyrozap This is probably actually a question for one of the other devs, but will weigh in in the hopes of getting you going - bear in mind my familiarity with 8051 is slight. So, while those values are hard-coded, I believe you should be able to edit them to whatever values you wish, i.e. click the "Edit" button in the Registers view and change the value of the stack pointer to set something other than the default, and in the Regions view, double-click and modify the start and stop addresses for the stack. I do think Tool Options entries for the default settings might be a nice feature. I will suggest that and add a ticket if folks agree.

[cyrozap]

@d-millar Unfortunately, it appears the registers are not editable until the emulation trace is started, and the exception is preventing the trace from being started. The same applies to the regions--they can't be edited until the trace is started, which can't happen due to the exception.

Regarding adding Tool Options entries for the default settings: If the default values were able to be modified, they should be restricted to the memory space available so that it won't be possible to pick invalid values, or if it would be possible, when it happens the tool should report an error to the user and not raise an exception. Also, I wouldn't mind if the emulator was able to automatically select some "safe" default values that would need to be modified for each new 8051 firmware, but in any case the tool needs to pick something that won't immediately raise an exception when clicking the "start new trace" button.

I think, ideally, the stack base and size values should be selected automatically based on the architecture, and if the default values for a particular architecture aren't available (e.g., because a user is using a custom processor module), it could be based on the size of the memory space the stack lives in. So in those cases, maybe the calculation could be, start the stack at the lower of either 0x1000 or a point 50% into the memory space (so a 64-bit system would have its stack start at 0x1000, a 32-bit system 0x1000, a 16-bit system 0x1000, and an 8-bit system 0x80), and make the stack size the smaller of either 0x4000 or 50% of the memory space (so a 64-bit system would have its stack size set to 0x4000, a 32-bit system 0x4000, a 16-bit system 0x4000, and an 8-bit system 0x80).

Maybe the minimum viable product (MVP) for this would be to select the stack base and size based on the heuristics described above, since as far as I can tell that should be a fairly straightforward fix. A later, more involved change could make it so that the default stack base and size could optionally be defined by each processor module (with a fallback to the heuristic if no defaults are provided), so that no changes would need to be made to Ghidra if a user wanted to emulate code using a custom processor module plugin.

[d-millar]

@cyrozap Got it - appreciate the clarification / discussion!

[nsadeveloper789]

I like this suggestion, @cyrozap. My approach was more heuristic to begin with, I just hadn't considered extremely small memory spaces. I neglected to check the size at all :/ . I think your heuristic makes sense, too. I was going for unoccupied space at the lower end of the address space.

At the very least, it shouldn't cause an exception. The worst-case fallback should just be sp=0 and leave it to the user to sort out.

If I understand your suggested heuristic:

• Address is the lesser of 0x1000 and the midpoint of the stack's address space.
• Size is the lesser of 0x4000 and half the stack's address space.

I'll add to this:

• If the region collides with an existing region, shift it up 0x1000 until it no longer collides, or space is exhausted.
• If space is exhausted, default to sp=0. Size is no longer relevant, because we're not creating a region in this case.

I agree, too, that adding a property to the compiler spec that gives these initial values would allow us to better match the actual behaviors/conventions of the target platform.