cortex-debug
cortex-debug copied to clipboard
Marus
Multi-chip debugging on two openocd instances not working
Describe the bug
I have a dual chip setup where I would like to debug both chips simultaneously. My setup is: RPi debug probe -> openocd -> chip 1 Master (cortex-M3) RPi debug probe -> openocd -> chip 2 Slave (cortex-M3)
I'm using a chained configuration. Both debugging chip 1 and chip 2 independently works (with below configuration), but in a chained configuration, the slave configuration does not start.
To Reproduce My launch.json looks as follows:
{
"version": "0.2.0",
"configurations": [
{
"name": "Master debug",
"cwd": "${workspaceFolder}",
"executable": "./code_master.axf",
"request": "attach",
"type": "cortex-debug",
"servertype": "openocd",
"configFiles": ["interface/cmsis-dap.cfg", "/home/openocd/setup.cfg"],
"openOCDLaunchCommands": ["cmsis_dap_serial E6616408435CC232"], //Master debug probe
"showDevDebugOutput": "raw",
"chainedConfigurations": {
"enabled": true,
"waitOnEvent": "postInit",
"detached": true,
"lifecycleManagedByParent": true,
"launches": [
{
"name": "Slave debug",
//"folder": "${workspaceFolder}"
"folder": ""
}
]
}
},
{
"name": "Slave debug",
"cwd": "${workspaceFolder}",
"executable": "./code_slave.axf",
"request": "attach",
"type": "cortex-debug",
"servertype": "openocd",
"configFiles": ["interface/cmsis-dap.cfg", "/home/openocd/setup.cfg"],
"openOCDLaunchCommands": ["cmsis_dap_serial E661640843551139"], //Slave debug probe
"showDevDebugOutput": "raw",
}
]
}
On running the chained configuration, I'm getting the following output:
** Debug Console **
For master, the debug console output looks as follows:
"configuration": {
"name": "Master debug",
"cwd": "/home/code",
"executable": "/home/code/code_master.axf",
"request": "attach",
"type": "cortex-debug",
"servertype": "openocd",
"configFiles": [
"interface/cmsis-dap.cfg",
"/home/openocd/setup.cfg"
],
"openOCDLaunchCommands": [
"cmsis_dap_serial E6616408435CC232"
],
"showDevDebugOutput": "raw",
"chainedConfigurations": {
"enabled": true,
"waitOnEvent": "postInit",
"detached": true,
"lifecycleManagedByParent": true,
"launches": [
{
"name": "Slave debug",
"folder": "/home/code",
"enabled": true,
"delayMs": 0,
"detached": true,
"waitOnEvent": "postInit",
"lifecycleManagedByParent": true
}
],
"delayMs": 0
},
"__configurationTarget": 6,
"gdbServerConsolePort": 55878,
"pvtAvoidPorts": [],
"debuggerArgs": [],
"swoConfig": {
"enabled": false,
"decoders": [],
"cpuFrequency": 0,
"swoFrequency": 0,
"source": "probe"
},
"rttConfig": {
"enabled": false,
"decoders": []
},
"graphConfig": [],
"preLaunchCommands": [],
"postLaunchCommands": [],
"preAttachCommands": [],
"postAttachCommands": [],
"preRestartCommands": [],
"postRestartCommands": [],
"preResetCommands": [],
"postResetCommands": [],
"searchDir": [],
"toolchainPrefix": "arm-none-eabi",
"extensionPath": "/home/rust/.vscode-server/extensions/marus25.cortex-debug-1.12.1",
"registerUseNaturalFormat": true,
"variableUseNaturalFormat": true,
"pvtVersion": "1.12.1",
"__sessionId": "369f7e9d-ea31-4bd5-8f19-8bc3820cb1f3",
"pvtShowDevDebugOutput": "raw"
}
Reading symbols from arm-none-eabi-objdump --syms -C -h -w /home/code/code_master.axf
Reading symbols from arm-none-eabi-nm --defined-only -S -l -C -p /home/code/code_master.axf
Launching GDB: arm-none-eabi-gdb -q --interpreter=mi2
{
"chainedConfigurations": {
"enabled": true,
"waitOnEvent": "postInit",
"detached": true,
"lifecycleManagedByParent": true,
"launches": [
{
"name": "Slave debug",
"folder": "/home/code",
"enabled": true,
"delayMs": 0,
"detached": true,
"waitOnEvent": "postInit",
"lifecycleManagedByParent": true
}
],
"delayMs": 0
}
}
1-gdb-version
Launching gdb-server: openocd -c "gdb_port 50000" -c "tcl_port 50001" -c "telnet_port 50002" -s /home/code -f /home/rust/.vscode-server/extensions/marus25.cortex-debug-1.12.1/support/openocd-helpers.tcl -f interface/cmsis-dap.cfg -f /home/openocd/setup.cfg -c "cmsis_dap_serial E6616408435CC232"
And for the failing slave:
"configuration": {
"name": "Slave debug",
"cwd": "/home/code",
"executable": "/home/code/code_slave.axf",
"request": "attach",
"type": "cortex-debug",
"servertype": "openocd",
"configFiles": [
"interface/cmsis-dap.cfg",
"/home/openocd/setup.cfg"
],
"openOCDLaunchCommands": [
"cmsis_dap_serial E661640843551139"
],
"showDevDebugOutput": "raw",
"__configurationTarget": 6,
"gdbServerConsolePort": 55878,
"pvtAvoidPorts": [
50000,
50001,
50002,
50003
],
"pvtParent": {
"name": "Master debug",
"cwd": "/home/code",
"executable": "/home/code/code_master.axf",
"request": "attach",
"type": "cortex-debug",
"servertype": "openocd",
"configFiles": [
"interface/cmsis-dap.cfg",
"/home/openocd/setup.cfg"
],
"openOCDLaunchCommands": [
"cmsis_dap_serial E6616408435CC232"
],
"showDevDebugOutput": "raw",
"chainedConfigurations": {
"enabled": true,
"waitOnEvent": "postInit",
"detached": true,
"lifecycleManagedByParent": true,
"launches": [
{
"name": "Slave debug",
"folder": "/home/code",
"enabled": true,
"delayMs": 0,
"detached": true,
"waitOnEvent": "postInit",
"lifecycleManagedByParent": true
}
],
"delayMs": 0
},
"__configurationTarget": 6,
"gdbServerConsolePort": 55878,
"pvtAvoidPorts": [],
"debuggerArgs": [],
"swoConfig": {
"enabled": false,
"decoders": [],
"cpuFrequency": 0,
"swoFrequency": 0,
"source": "probe"
},
"rttConfig": {
"enabled": false,
"decoders": []
},
"graphConfig": [],
"preLaunchCommands": [],
"postLaunchCommands": [],
"preAttachCommands": [],
"postAttachCommands": [],
"preRestartCommands": [],
"postRestartCommands": [],
"preResetCommands": [],
"postResetCommands": [],
"searchDir": [],
"toolchainPrefix": "arm-none-eabi",
"extensionPath": "/home/rust/.vscode-server/extensions/marus25.cortex-debug-1.12.1",
"registerUseNaturalFormat": true,
"variableUseNaturalFormat": true,
"pvtVersion": "1.12.1",
"__sessionId": "369f7e9d-ea31-4bd5-8f19-8bc3820cb1f3",
"pvtShowDevDebugOutput": "raw",
"gdbPath": "arm-none-eabi-gdb",
"numberOfProcessors": 1,
"targetProcessor": 0,
"pvtPorts": {
"gdbPort": 50000,
"tclPort": 50001,
"telnetPort": 50002,
"swoPort": 50003
}
},
"pvtMyConfigFromParent": {
"name": "Slave debug",
"folder": "/home/code",
"enabled": true,
"delayMs": 0,
"detached": true,
"waitOnEvent": "postInit",
"lifecycleManagedByParent": true
},
"chainedConfigurations": {
"enabled": false
},
"debuggerArgs": [],
"swoConfig": {
"enabled": false,
"decoders": [],
"cpuFrequency": 0,
"swoFrequency": 0,
"source": "probe"
},
"rttConfig": {
"enabled": false,
"decoders": []
},
"graphConfig": [],
"preLaunchCommands": [],
"postLaunchCommands": [],
"preAttachCommands": [],
"postAttachCommands": [],
"preRestartCommands": [],
"postRestartCommands": [],
"preResetCommands": [],
"postResetCommands": [],
"toolchainPrefix": "arm-none-eabi",
"extensionPath": "/home/rust/.vscode-server/extensions/marus25.cortex-debug-1.12.1",
"registerUseNaturalFormat": true,
"variableUseNaturalFormat": true,
"pvtVersion": "1.12.1",
"__sessionId": "1ca0de02-dc87-44ea-b249-34f87eac3dc2",
"pvtShowDevDebugOutput": "raw"
}
Reading symbols from arm-none-eabi-objdump --syms -C -h -w /home/code/code_slave.axf
Reading symbols from arm-none-eabi-nm --defined-only -S -l -C -p /home/code/code_slave.axf
Launching GDB: arm-none-eabi-gdb -q --interpreter=mi2
1-gdb-version
I suspect it is something with openocd trying to use the same ports for both gdb instances? Any help would be greatly appreciated!
Manually spinning up two openocd instances and using the "servertype": "external" setting works flawlessly
What you are doing is multi-chip debug. Not multi-core. Chained configurations are really needed for multi-core. If you have totally two separate chips,
- Make sure the two launch configs work stand alone
- Use compounds built into VSCode already. See https://code.visualstudio.com/docs/editor/debugging#_compound-launch-configurations
However, you can also used chained configurations. Note that in the slave, we have not even launched openocd for the second instance. We also passed information to the slave to avoid ports used by the master. The problem I see is that gdb has not even started.... since there was no response for the 1-gdb-version command.
Does the slave work standalone properly?
Chained configurations are really needed for multi-core.
Thanks for this clarification. I was looking for an easy way to spin up multiple sessions within VSCode and have some global events/actions available. Compounding might work just as well, I will give it a try!
Does the slave work standalone properly?
It does. Clicking on Slave Debug will connect.
configuration": {
"name": "Slave debug",
"cwd": "/home/code",
"executable": "/home/code/code_slave.axf",
"request": "attach",
"type": "cortex-debug",
"servertype": "openocd",
"configFiles": [
"interface/cmsis-dap.cfg",
"/home/openocd/setup.cfg"
],
"openOCDLaunchCommands": [
"cmsis_dap_serial E661640843551139"
],
"showDevDebugOutput": "raw",
"__configurationTarget": 6,
"gdbServerConsolePort": 55878,
"pvtAvoidPorts": [],
"chainedConfigurations": {
"enabled": false
},
"debuggerArgs": [],
"swoConfig": {
"enabled": false,
"decoders": [],
"cpuFrequency": 0,
"swoFrequency": 0,
"source": "probe"
},
"rttConfig": {
"enabled": false,
"decoders": []
},
"graphConfig": [],
"preLaunchCommands": [],
"postLaunchCommands": [],
"preAttachCommands": [],
"postAttachCommands": [],
"preRestartCommands": [],
"postRestartCommands": [],
"preResetCommands": [],
"postResetCommands": [],
"searchDir": [],
"toolchainPrefix": "arm-none-eabi",
"extensionPath": "/home/rust/.vscode-server/extensions/marus25.cortex-debug-1.12.1",
"registerUseNaturalFormat": true,
"variableUseNaturalFormat": true,
"pvtVersion": "1.12.1",
"__sessionId": "bca4060d-392f-4e39-a441-76e3cccca9f3",
"pvtShowDevDebugOutput": "raw"
}
Reading symbols from arm-none-eabi-objdump --syms -C -h -w /home/code/code_slave.axf
Reading symbols from arm-none-eabi-nm --defined-only -S -l -C -p /home/code/code_slave.axf
Launching GDB: arm-none-eabi-gdb -q --interpreter=mi2
1-gdb-version
Launching gdb-server: openocd -c "gdb_port 50000" -c "tcl_port 50001" -c "telnet_port 50002" -s /home/code/ -f /home/rust/.vscode-server/extensions/marus25.cortex-debug-1.12.1/support/openocd-helpers.tcl -f interface/cmsis-dap.cfg -f /home/code/setup.cfg -c "cmsis_dap_serial E661640843551139"
Finished reading symbols from objdump: Time: 58 ms
I can't spot anything obviously wrong. I'll give compounds a try, maybe I'll run into the same issue or can spot a difference
Notice that when you were doing it as a chained configuration, the last this we saw was asking gdb what version it was. It never returned. This usually happens with a bad gdb installation -- as in some libraries are missing. Typically curses or python. Unless gdb communication is confirmed, we don't even start openocd.
But, your first launch went through and your standalone launches work.
One issue you may run into with compounds is a collision of TCP ports unless you can delay the start of the second session.
Just for good measure I tried with an older GDB version
GNU gdb (GNU Toolchain for the Arm Architecture 11.2-2022.02 (arm-11.14)) 11.2.90.20220202-git, but same issue. The second session does not return anything for 1-gdb-version. Doing this manually for two GDB instances works flawlessly though.
One issue you may run into with compounds is a collision of TCP ports unless you can delay the start of the second session.
I faced this issue, but was able to make a compound session run successfully by introducing a SleepDelay task as described here
Still odd as to why the chained configuration approach should not work. Its not a great blocker, but compounds with a delay is a little less nice I think. :)
Thanks for your input so far. Can you think of a way to proceed with this, maybe more things I could try? If not, feel free to close this ticket, as the issue might well be on my side somewhere.
Can you try reversing the Master/Slave launch sequence? Also, check the Process Viewer/Task Manager or whatever to see if both instances of gdb are running when it is hung like that.
Grabbing at straws, you can also try changing the waitOnEvent to something else. Doubt it will help at all.
Can you try reversing the Master/Slave launch sequence?
I did this by reversing the debugger serial numbers and the path to the executable. No change in behavior. The first configuration (previously the second) starts up fine, and the second one fails to spin up a gdb client.
Also, check the Process Viewer/Task Manager or whatever to see if both instances of gdb are running
After this sequence, only one instance of gdb is running (and one instance of openocd). I can manually start further instances by running arm-none-eabi-gdb -q --interpreter=mi2 in a console, and that works flawlessly again and shows in the task manager.
you can also try changing the waitOnEvent to something else
Didn't change behavior, unfortunately.
Would be interesting to know if this behavior is reproducible with any two debuggers started in the same way.
This is how my "/home/openocd/setup.cfg" looks like:
# Your SWD adapter config
bindto 0.0.0.0
transport select swd
adapter speed 1000
swd newdap chip cpu -enable
dap create chip.dap -chain-position chip.cpu
target create chip.cpu cortex_m -dap chip.dap
#init
#dap info
I thought I might run into trouble by naming both the dap's "chip.dap". So I created a setup1.cfg for the slave where I named everything chip1.dap/chip1.cpu, etc. Unfortunately, also no change in behavior.