Linker problems with ARM926 core / AT91SAM9260 BSP

[WebDust21]

SYSTEM SETUP:

• Ubuntu 22.04 LTS x64
• utilizing arm-none-eabi toolchain

I have recently started modifying the AT91SAM9260 RT-Thread BSP for an AT91SAM9263, and came across some fatal issues that will prevent use of the AT91SAM9260 RT-Thread BSP. I am not very good at GitHub, nor do I know the proper way to fix the problems, so I am creating an issue instead of a pull request.

The major problem has to do with the linker script, as well as the ARM926 CPU defs. After successfully compiling the project, I checked the disassembly with:

arm-none-eabi-objdump -d -g rtthread.elf > rtthread.lst

An ARM binary must start with a vector interrupt table. The RT-Thread AT91SAM9260 binary file does not, check this disassembly:

./rtthread-at91sam9260.elf:     file format elf32-littlearm


Disassembly of section .text:

20000000 <_init>:
20000000:	e1a0c00d 	mov	ip, sp
20000004:	e92ddff8 	push	{r3, r4, r5, r6, r7, r8, r9, sl, fp, ip, lr, pc}
20000008:	e24cb004 	sub	fp, ip, #4

2000000c <list_fd>:
2000000c:	e92d4ff8 	push	{r3, r4, r5, r6, r7, r8, r9, sl, fp, lr}
20000010:	eb003da5 	bl	2000f6ac <rt_enter_critical>
20000014:	e59f515c 	ldr	r5, [pc, #348]	; 20000178 <list_fd+0x16c>
20000018:	e59f015c 	ldr	r0, [pc, #348]	; 2000017c <list_fd+0x170>
2000001c:	eb003795 	bl	2000de78 <rt_kprintf>
20000020:	e59f0158 	ldr	r0, [pc, #344]	; 20000180 <list_fd+0x174>
20000024:	eb003793 	bl	2000de78 <rt_kprintf>
20000028:	e5953000 	ldr	r3, [r5]
2000002c:	e3530000 	cmp	r3, #0

Perhaps unsurprisingly, the binary will not work.

I did check the output of the "qemu-vexpress-a9" (as an example that works perfectly in QEMU), and as expected, it starts with a vector table:

./rtthread.elf:     file format elf32-littlearm


Disassembly of section .text:

60010000 <__text_start>:
60010000:	e59ff018 	ldr	pc, [pc, #24]	; 60010020 <_vector_reset>
60010004:	e59ff018 	ldr	pc, [pc, #24]	; 60010024 <_vector_undef>
60010008:	e59ff018 	ldr	pc, [pc, #24]	; 60010028 <_vector_swi>
6001000c:	e59ff018 	ldr	pc, [pc, #24]	; 6001002c <_vector_pabt>
60010010:	e59ff018 	ldr	pc, [pc, #24]	; 60010030 <_vector_dabt>
60010014:	e59ff018 	ldr	pc, [pc, #24]	; 60010034 <_vector_resv>
60010018:	e59ff018 	ldr	pc, [pc, #24]	; 60010038 <_vector_irq>
6001001c:	e59ff018 	ldr	pc, [pc, #24]	; 6001003c <_vector_fiq>

60010020 <_vector_reset>:
60010020:	6005f280 	.word	0x6005f280

60010024 <_vector_undef>:
60010024:	60116fe0 	.word	0x60116fe0

60010028 <_vector_swi>:
60010028:	6002d1e4 	.word	0x6002d1e4

6001002c <_vector_pabt>:
6001002c:	60117040 	.word	0x60117040

60010030 <_vector_dabt>:
60010030:	601170a0 	.word	0x601170a0

60010034 <_vector_resv>:
60010034:	60117100 	.word	0x60117100

Digging into the issue, I found multiple problems.

The vector table for the ARM926 core is found in "rt-thread-master/libcpu/arm/arm926/start_gcc.S" Checking the build messages, we do see that "start_gcc.S" is being compiled:

CC build/kernel/libcpu/arm/arm926/mmu.o
CC build/kernel/libcpu/arm/arm926/stack.o
**AS build/kernel/libcpu/arm/arm926/start_gcc.o**
CC build/kernel/libcpu/arm/arm926/trap.o
CC build/kernel/libcpu/arm/common/backtrace.o

Looking into "start_gcc.S", we find the linker points:

 /*
 ***************************************
 * Interrupt vector table
 ***************************************
 */
.section .vectors
.code 32

.global start		//.global system_vectors
start: //system_vectors:
    ldr pc, _vector_reset
    ldr pc, _vector_undef
    ldr pc, _vector_swi
    ldr pc, _vector_pabt
    ldr pc, _vector_dabt
    ldr pc, _vector_resv
    ldr pc, _vector_irq
    ldr pc, _vector_fiq

Of particular interest is the line, ".section .vectors". Time to look into the BSP linker script, "rt-thread-master/bsp/at91/at91sam9260/link_scripts/at91sam9260_ram.ld":

OUTPUT_FORMAT("elf32-littlearm", "elf32-littlearm", "elf32-littlearm")
OUTPUT_ARCH(arm)
ENTRY(start)
SECTIONS
{
    . = 0x20000000;

    . = ALIGN(4);
    .text : 
    {
        *(.init)
        *(.text)
        *(.gnu.linkonce.t*)

Wait a minute, the first entry is a vector titled ".init". But in "start_gcc.S", it's supposed to be ".vectors".

So we fix the linker script to read as follows:

OUTPUT_FORMAT("elf32-littlearm", "elf32-littlearm", "elf32-littlearm")
OUTPUT_ARCH(arm)
ENTRY(start)
SECTIONS
{
    . = 0x20000000;

    . = ALIGN(4);
    .text : 
    {
        *(.vectors)
        *(.text)
        *(.gnu.linkonce.t*)

Compiling now returns some errors:

LINK rtthread-at91sam9260.elf
build/kernel/libcpu/arm/arm926/start_gcc.o: In function `vector_resv':
(.text+0x220): undefined reference to `__bss_start'
build/kernel/libcpu/arm/arm926/start_gcc.o: In function `vector_resv':
(.text+0x224): undefined reference to `__bss_end'
collect2: error: ld returned 1 exit status
scons: *** [rtthread-at91sam9260.elf] Error 1
scons: building terminated because of errors.

It turns out that this is a nomenclature difference between "start_gcc.S" lines 135-136 and the linker script:

    /* clear bss */
    mov     r0, #0
    ldr     r1, =__bss_start
    ldr     r2, =__bss_end

and the linker script (lines 78-80):

    . = ALIGN(4);
    __bss_start__ = .;
    .bss : { *(.bss)}
    __bss_end__ = .;

Editing the "start_gcc.S" lines 135-136 to match the linker script solves the problem. I am not sure whether "start_gcc.S" or the linker script is wrong here...but after making them match, now it compiles successfully without errors:

LINK rtthread-at91sam9260.elf
arm-none-eabi-objcopy -O binary rtthread-at91sam9260.elf rtthread.bin
arm-none-eabi-objdump -d -g rtthread-at91sam9260.elf > rtthread.lst
arm-none-eabi-size rtthread-at91sam9260.elf
   text    data     bss     dec     hex filename
  99009    4332   35584  138925   21ead rtthread-at91sam9260.elf
scons: done building targets.

And better yet, checking the disassembly LST file, we find the following:

rtthread-at91sam9260.elf:     file format elf32-littlearm


Disassembly of section .text:

20000000 <start>:
20000000:	e59ff018 	ldr	pc, [pc, #24]	; 20000020 <_vector_reset>
20000004:	e59ff018 	ldr	pc, [pc, #24]	; 20000024 <_vector_undef>
20000008:	e59ff018 	ldr	pc, [pc, #24]	; 20000028 <_vector_swi>
2000000c:	e59ff018 	ldr	pc, [pc, #24]	; 2000002c <_vector_pabt>
20000010:	e59ff018 	ldr	pc, [pc, #24]	; 20000030 <_vector_dabt>
20000014:	e59ff018 	ldr	pc, [pc, #24]	; 20000034 <_vector_resv>
20000018:	e59ff018 	ldr	pc, [pc, #24]	; 20000038 <_vector_irq>
2000001c:	e59ff018 	ldr	pc, [pc, #24]	; 2000003c <_vector_fiq>

20000020 <_vector_reset>:
20000020:	20008e44 	.word	0x20008e44

20000024 <_vector_undef>:
20000024:	20008fbc 	.word	0x20008fbc

20000028 <_vector_swi>:
20000028:	20008f94 	.word	0x20008f94

The binary starts with ARM vectors! Now it can actually run!

My port of the AT91SAM9260 BSP to the AT91SAM9263 with these fixes (as well as hundreds of other changes to fix the register differences between the 2 CPU types) booted and ran perfectly on target hardware:

U-Boot> tftp 0x20000000 "rtthread.bin"
macb0: link up, 100Mbps full-duplex (lpa: 0xcde1)
Using macb0 device
TFTP from server 192.168.1.102; our IP address is 192.168.1.250
Filename 'rtthread.bin'.
Load address: 0x20000000
Loading: *########
done
Bytes transferred = 103364 (193c4 hex)
U-Boot> go 0x20000000
## Starting application at 0x20000000 ...
pit_rate=6221647HZ
PIT_MR=0x0300184d

 \ | /
- RT -     Thread Operating System
 / | \     5.0.0 build Nov  4 2022 19:45:44
 2006 - 2022 Copyright by RT-Thread team
msh />help
RT-Thread shell commands:
list_fd          - list file descriptor
pin              - pin [option]
list             - list objects
list_device      - list device in system
list_timer       - list timer in system
list_mempool     - list memory pool in system
list_msgqueue    - list message queue in system
list_mailbox     - list mail box in system
list_mutex       - list mutex in system
list_event       - list event in system
list_sem         - list semaphore in system
list_thread      - list thread
version          - show RT-Thread version information
clear            - clear the terminal screen
free             - Show the memory usage in the system.
ps               - List threads in the system.
help             - RT-Thread shell help.
tail             - print the last N - lines data of the given file
echo             - echo string to file
df               - disk free
umount           - Unmount device from file system
mount            - mount <device> <mountpoint> <fstype>
mkfs             - format disk with file system
mkdir            - Create the DIRECTORY.
pwd              - Print the name of the current working directory.
cd               - Change the shell working directory.
rm               - Remove(unlink) the FILE(s).
cat              - Concatenate FILE(s)
mv               - Rename SOURCE to DEST.
cp               - Copy SOURCE to DEST.
ls               - List information about the FILEs.
shutdown         - shutdown the system
reset            - restart the system

I am presenting these findings in hope that they will be valuable to the developer team, for diagnostics and fixing the BSP issues.

[BernardXiong]

Thank you for your feed back and correct ld script. I will fix it soon.