AVR: java.lang.IllegalArgumentException: Address are in different spaces code != codebyte

[ghost]

While executing a program search (irrelevant but: PORTn_OUT constructs), the following error comes up:

Address are in different spaces code != codebyte
java.lang.IllegalArgumentException: Address are in different spaces code != codebyte
	at ghidra.program.model.address.AbstractAddressSpace.subtract(AbstractAddressSpace.java:292)
	at ghidra.program.model.address.GenericAddressSpace.subtract(GenericAddressSpace.java:21)
	at ghidra.program.model.address.GenericAddress.subtract(GenericAddress.java:144)
	at ghidra.program.database.function.FunctionManagerDB.getReferencedVariable(FunctionManagerDB.java:1273)
	at ghidra.program.model.listing.CodeUnitFormat.performRegisterMarkup(CodeUnitFormat.java:328)
	at ghidra.program.model.listing.CodeUnitFormat.getOperandRepresentationList(CodeUnitFormat.java:256)
	at ghidra.program.model.listing.CodeUnitFormat.getOperandRepresentationString(CodeUnitFormat.java:179)
	at ghidra.app.plugin.core.searchtext.databasesearcher.InstructionMnemonicOperandFieldSearcher.getOperandStrings(InstructionMnemonicOperandFieldSearcher.java:166)
	at ghidra.app.plugin.core.searchtext.databasesearcher.InstructionMnemonicOperandFieldSearcher.findMatchesForCurrentAddress(InstructionMnemonicOperandFieldSearcher.java:90)
	at ghidra.app.plugin.core.searchtext.databasesearcher.InstructionMnemonicOperandFieldSearcher.advance(InstructionMnemonicOperandFieldSearcher.java:82)
	at ghidra.app.plugin.core.searchtext.databasesearcher.ProgramDatabaseFieldSearcher.doAdvance(ProgramDatabaseFieldSearcher.java:52)
	at ghidra.app.plugin.core.searchtext.databasesearcher.ProgramDatabaseFieldSearcher.getNextSignificantAddress(ProgramDatabaseFieldSearcher.java:69)
	at ghidra.app.plugin.core.searchtext.databasesearcher.ProgramDatabaseSearcher.findNextSignificantAddress(ProgramDatabaseSearcher.java:139)
	at ghidra.app.plugin.core.searchtext.databasesearcher.ProgramDatabaseSearcher.search(ProgramDatabaseSearcher.java:101)
	at ghidra.app.plugin.core.searchtext.AbstractSearchTableModel.doLoad(AbstractSearchTableModel.java:65)
	at docking.widgets.table.threaded.IncrementalLoadJob.doExecute(IncrementalLoadJob.java:82)
	at docking.widgets.table.threaded.IncrementalLoadJob.run(IncrementalLoadJob.java:61)
	at ghidra.util.worker.AbstractWorker$JobCallback.process(AbstractWorker.java:133)
	at ghidra.util.worker.AbstractWorker$JobCallback.process(AbstractWorker.java:123)
	at generic.concurrent.ConcurrentQ$CallbackCallable.call(ConcurrentQ.java:658)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at generic.concurrent.FutureTaskMonitor.run(FutureTaskMonitor.java:76)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

---------------------------------------------------
Build Date: 2023-Feb-08 1242 EST
Ghidra Version: 10.2.3
Java Home: xxx
JVM Version: xxx
OS: xxx

[ghidra1]

For AVR8: Functions should not be defined in the codebyte space. I believe all code and functions should be defined in the code space only. The byte-mapped codebyte space exists only to facilitate data access.

[ghidra1]

Are you able to share you code (*.gzf) so we can better understand your situation which has caused the use of code and codebyte space to become entangled for the function which triggered this exception?

[ghost]

I believe all code and functions should be defined in the code space only. The byte-mapped codebyte space exists only to facilitate data access.

Agreed, and the issue resolves if those functions are cleared from codebyte, but the issue at the core is improper handling of segments, besides the well known (for XMEGA at least) issue with boot-time code/data loading (bss zeroing and data copy). We live through amazing modern times with AVR8 being capable of self-modification... and these quirks are the result of that.

Are you able to share you code (*.gzf) so we can better understand your situation which has caused the use of code and codebyte space to become entangled for the function which triggered this exception?

Nope. I might be able to unearth some sample code compiled to simulate this but I don't have anything that I am allowed to share or is not encumbered by some IP restrictions.

[emteere]

Would be great to have an example to show what you are running into.

Also sounds like you are trying to automate the boot up code that would copy different parts of data memory into ram.

Are you also trying to handle overlays, where different code is loaded at the same memory at any given time? That might be complicated with the code/codebyte spaces. I've never tried to set that up.

The current code/codebyte is sub-optimal. It was done in order to do correct calculations and interpretation of constants. We've been discussing better ways of getting all references to go to the same location, but keep byte addresses and word addresses separate.

[ghost]

Sorry for the delay.

Also sounds like you are trying to automate the boot up code that would copy different parts of data memory into ram.

Yes, to some extent. Ideally I would like to automate the entire bootstrapping (should be relatively easy with some clever processing of the copy loops, the existent script can use some improvement). I don't know how often I will deal with AVR8 projects but at the moment the processor could benefit from automatic memory map configuration without the codebyte hack.

As for the overlays, I just manually create eeprom and the like from real memory dumps.

We've been discussing better ways of getting all references to go to the same location, but keep byte addresses and word addresses separate.

This seems like the way to go. There is a separate issue for the CONCAT11 (aka LOW/HI byte) mess.

For example:

        uVar11 = *(undefined2 *)
                  (CONCAT11(PTR_to_struct._1_1_,
                            PTR_to_struct._0_1_) + SOME_OFFSET);

With PTR_to_struct properly defined as a pointer to a well-defined structure data type (which has function typedefs and so on), these should no longer be processed like that. If + SOME_OFFSET resolves to a specific field in the structure, it should be presented as such to the user.

I'm probably missing something I could do to beautify these, but it does seem like we have deeper issues with the decompiler. The quality of life improvement in properly handling those will be astronomical.

At the moment we also have issues with how AVR handles some data references (ex. it relies on subtraction operations for things like index or table lookups, so we look for the functions using such tables by searching for insn patterns crafted by hand with the correct complement for the -known- target table).

I will find that issue and link it here, just for reference.

[ghidra1]

This is related to #2577.