Creating function in different program silently fails

[Martmists-GH]

Describe the bug Creating a function in a linked program silently fails.

To Reproduce Code:

open class CROLinkingScriptImpl : GhidraScript() {
    private fun getSegmentName(segment: Int) = when (segment) {
        0 -> ".text"
        1 -> ".rodata"
        2 -> ".data"
        else -> throw IllegalArgumentException("Invalid segment $segment")
    }

    private fun tryLocateCRO(moduleName: String): Library? {
        // Find the file based on module name
        val library = currentProgram.externalManager.getExternalLibrary(moduleName)
        if (library.associatedProgramPath == null) {
            val fileName = if (moduleName == "|static|") "static.crs" else "$moduleName.cro"
            val file = currentProgram.domainFile.parent.getFile(fileName) ?: return null
            currentProgram.externalManager.setExternalPath(moduleName, file.pathname, false)
        }
        return currentProgram.externalManager.getExternalLibrary(moduleName)
    }

    override fun run() {
        val provider = MemoryByteProvider.createMemoryBlockByteProvider(currentProgram.memory, currentProgram.memory.getBlock("header") ?: return)
        val tableProvider = MemoryByteProvider.createMemoryBlockByteProvider(currentProgram.memory, currentProgram.memory.getBlock("tables") ?: return)

        val header = provider.getInputStream(0).reader {
            read<CRO0Header>()
        }

        tableProvider.getInputStream(0).reader {
            seek(header.importModuleTableOffset - header.segmentTableOffset)
            val modules = readList<CRO0Header.ImportModuleTableEntry>(header.importModuleTableNum)

            for (mod in modules) {
                seek(mod.nameOffset - header.segmentTableOffset)
                val moduleName = readNullTerminatedString()

                val library = tryLocateCRO(moduleName) ?: throw IllegalStateException("Library $moduleName not assigned!")
                val libraryProgram = library.symbol.program
                val api = FlatProgramAPI(libraryProgram, monitor)

                seek(mod.anonymousHeadOffset - header.segmentTableOffset)
                val anonymousImports = readList<CRO0Header.AnonymousImportTableEntry>(mod.anonymousImportNum)

                for (import in anonymousImports) {
                    seek(import.segmentOffset - header.segmentTableOffset)
                    val importSegment = import.segmentOffset and 0xF
                    val importOffset = import.segmentOffset shr 4

                    val block = libraryProgram.memory.getBlock(getSegmentName(importSegment))
                    val addr = block.start.add(importOffset.toLong())

                    val sym = currentProgram.symbolTable.getSymbols("${moduleName}_anonymous_${importSegment}_${importOffset.toString(16)}").first()
                    val importedFunc = currentProgram.functionManager.getFunctionAt(sym.address)
                    if (api.getFunctionAt(addr) == null) {
                        api.createFunction(addr, "${moduleName}_anonymous_${importSegment}_${importOffset.toString(16)}")
                    }
                    val externalFunction = api.getFunctionAt(addr)!!  // Error: api.getFunctionAt returned null
                    if (externalFunction.name != "${moduleName}_anonymous_${importSegment}_${importOffset.toString(16)}") {
                        importedFunc.setName(externalFunction.name, SourceType.ANALYSIS)
                    }
                }
            }
        }
    }
}

Expected behavior The function gets created and getFunctionAt returns said function

Environment (please complete the following information):

• OS: Arch Linux
• Java Version: 11
• Ghidra Version: 10.1.5
• Ghidra Origin: AUR

Additional context This behavior happens both as a script as well as when implemented as analyzer.

[dev747368]

Does putting a api.start() (to start a db transaction) right after val api = FlatProgramAPI...., and then a api.end() somewhere at the bottom help things out?

[Martmists-GH]

It does not, I still get the NPE

[ghidra1]

While you definitely need library transaction independent of currentProgram you have a few places where you assume an expected value will be returned and not null. Need to add checks for this. Examples:

• val block = libraryProgram.memory.getBlock(getSegmentName(importSegment)) *Block may not be found - there may also be more than one block with the same name and the first one found is returned
• currentProgram.symbolTable.getSymbols("${moduleName}_anonymous_${importSegment}_${importOffset.toString(16)}") *Symbol may not be found - there may also be more than one. Some loaders have a tendancy to produce multiple symbols for the same import. This issue has been address for ELF and PE but still exists in other loaders. What loader/importer was used?

What line of code produces your NPE?

[Martmists-GH]

Block may not be found - there may also be more than one block with the same name and the first one found is returned

I can ensure the block exists and there is only one.

Symbol may not be found - there may also be more than one.

I can ensure this symbol exists and is unique in the program.

This issue has been address for ELF and PE but still exists in other loaders. What loader/importer was used?

This is using a custom implemented loader, which ensures to only define unique symbols once.

What line of code produces your NPE?

// In Kotlin, `!!` is shorthand for "error if this is null"
val externalFunction = api.getFunctionAt(addr)!!  // Error: api.getFunctionAt returned null

[ghidra1]

The api.createFunction may be failing and will return null when it does. You may have to delve into the CreateFunctionCmd - stepping through in debug may be the easiest way to troubleshoot. You may also try using it directly so that you can dig out the status message if the command fails. The FlatProgramAPI.createFunction method does not log the failure if one occurs.

Hopefully the following pseudo-code helps:

CreateFunctionCmd cmd = new CreateFunctionCmd(name, entryPoint, null,
			name != null ? SourceType.USER_DEFINED : SourceType.DEFAULT);
if (!cmd.applyTo(currentProgram, monitor)) {
    error(cmd.getStatusMsg());
}
else {
   return cmd.getFunction();
}

[ghidra1]

I am going to assume you were able to diagnose your script failure and close this ticket. Feel free to post any follow-up comments if you like. We can always re-open if neccessary.