linux icon indicating copy to clipboard operation
linux copied to clipboard

Rpi 6.17.y based on mainline dt

Open 6by9 opened this issue 4 months ago • 9 comments

I'm wanting comments on approach more than a full review.

At present there is a fair amount of board specific stuff mixed in to bcm2712-ds.dtsi, which is why CM5 is disabled just to reduce the number of errors thrown out as I was fixing it up. I'm suggesting we have bcm2712-rpi-5-b-ds.dtsi, bcm2712-rpi-500-ds.dtsi, bcm2712-rpi-cm5-ds.dtsi, etc, to contain the board specific stuff, and those will #include "bcm2712-ds.dtsi" which will define the devices. Hopefully the only changes to the main upstream files will then be to add the include that downstream one, therefore making Dom's life easier. As more devices get upstream support they will move from -ds.dtsi to the main files.

I'm aware that I'm missing HDMI audio dma configuration at present. I'm not seeing any other major omissions from the kernel logs, but haven't tested everything.

@pelwell - you'd reverted the clock driver to the downstream one. I know it's missing RP1_CLK_SDIO_TIMER and RP1_CLK_SDIO_ALT_SRC, but what else do you think it is missing? (I'll try and derive the entries for those two clocks now)

6by9 avatar Aug 19 '25 17:08 6by9

The Pi 5 dts file built from this tree references the following clocks:

RP1_CLK_ADC
RP1_CLK_AUDIO_OUT
RP1_CLK_DMA
RP1_CLK_DPI
RP1_CLK_ETH
RP1_CLK_ETH_TSU
RP1_CLK_GP0
RP1_CLK_GP1
RP1_CLK_GP2
RP1_CLK_GP3
RP1_CLK_GP4
RP1_CLK_GP5
RP1_CLK_I2S
RP1_CLK_MIPI0_CFG
RP1_CLK_MIPI0_DPI
RP1_CLK_MIPI0_DSI_BYTECLOCK
RP1_CLK_MIPI1_CFG
RP1_CLK_MIPI1_DPI
RP1_CLK_MIPI1_DSI_BYTECLOCK
RP1_CLK_PWM0
RP1_CLK_PWM1
RP1_CLK_SDIO_TIMER
RP1_CLK_SLOW_SYS
RP1_CLK_SYS
RP1_CLK_UART
RP1_CLK_VEC

And clk-rp1.c defines the following clocks:

RP1_CLK_ETH
RP1_CLK_ETH_TSU
RP1_CLK_SYS

We can ignore the GP clocks for now, but quite a few of the others are important.

pelwell avatar Aug 20 '25 09:08 pelwell

I think https://lore.kernel.org/linux-arm-kernel/17e5c6e0c085cfa0bf4b63b639cdc92c6a4c1418.1750714412.git.andrea.porta@suse.com/ (which is in linux-next ie 6.18) should add all of those for us. I'll cherry-pick it.

6by9 avatar Aug 20 '25 14:08 6by9

All the ethernet patches for RP1 have at least got R-b tags, so it may be worth backporting the official versions of those as well. https://lore.kernel.org/linux-arm-kernel/[email protected]/

6by9 avatar Aug 20 '25 14:08 6by9

Branch updated with the updated clock driver. It appears to include all the clocks you've listed.

clk_summary gives me:

pi@pi5-nvme:~ $ cat /sys/kernel/debug/clk/clk_summary 
                                 enable  prepare  protect                                duty  hardware                            connection
   clock                          count    count    count        rate   accuracy phase  cycle    enable   consumer                         id
---------------------------------------------------------------------------------------------------------------------------------------------
 fw-clk-disp                         1       1        0        0           0          0     50000      Y   107c580000.hvs                  disp                     
                                                                                                           deviceless                      no_connection_id         
 fw-clk-vec                          0       0        0        0           0          0     50000      Y   deviceless                      no_connection_id         
 fw-clk-pixel-bvb                    1       1        0        324000000   0          0     50000      Y   107c706400.hdmi                 bvb                      
                                                                                                           107c701400.hdmi                 bvb                      
                                                                                                           deviceless                      no_connection_id         
 fw-clk-m2mc                         3       3        0        599940000   0          0     50000      Y   107c706400.hdmi                 hdmi                     
                                                                                                           107c701400.hdmi                 hdmi                     
                                                                                                           deviceless                      no_connection_id         
 fw-clk-hevc                         0       0        0        500000000   0          0     50000      Y   1000800000.codec                no_connection_id         
                                                                                                           deviceless                      no_connection_id         
 fw-clk-pixel                        0       0        0        0           0          0     50000      Y   deviceless                      no_connection_id         
 fw-clk-isp                          0       0        0        500000000   0          0     50000      Y   1000880000.pisp_be              no_connection_id         
                                                                                                           deviceless                      no_connection_id         
 fw-clk-v3d                          1       1        0        500000000   0          0     50000      Y   1002000000.v3d                  no_connection_id         
                                                                                                           deviceless                      no_connection_id         
 fw-clk-core                         1       1        0        500000000   0          0     50000      Y   107c580000.hvs                  core                     
                                                                                                           deviceless                      no_connection_id         
 fw-clk-arm                          0       0        0        1500000000  0          0     50000      Y   cpu0                            no_connection_id         
                                                                                                           cpu0                            no_connection_id         
                                                                                                           deviceless                      no_connection_id         
 rp1-xosc                            5       5        0        50000000    0          0     50000      Y   deviceless                      no_connection_id         
    clksrc_mipi1_dsi_byteclk         0       0        0        0           0          0     50000      Y      deviceless                      no_connection_id         
    clksrc_mipi0_dsi_byteclk         0       0        0        0           0          0     50000      Y      deviceless                      no_connection_id         
    clk_gp5                          0       0        0        50000000    0          0     50000      Y      deviceless                      no_connection_id         
    clk_gp4                          0       0        0        50000000    0          0     50000      Y      deviceless                      no_connection_id         
    clk_gp3                          0       0        0        50000000    0          0     50000      Y      deviceless                      no_connection_id         
    clk_gp0                          0       0        0        50000000    0          0     50000      Y      deviceless                      no_connection_id         
    clk_sdio_timer                   0       0        0        1000000     0          0     50000      Y      deviceless                      no_connection_id         
       clk_gp1                       0       0        0        1000000     0          0     50000      Y         deviceless                      no_connection_id         
    clk_adc                          0       0        0        50000000    0          0     50000      Y      deviceless                      no_connection_id         
    clk_eth_tsu                      1       1        0        50000000    0          0     50000      Y      1f00100000.ethernet             tsu_clk                  
                                                                                                              1f00100000.ethernet             tsu_clk                  
                                                                                                              deviceless                      no_connection_id         
    clk_mipi1_cfg                    0       0        0        50000000    0          0     50000      Y      deviceless                      no_connection_id         
    clk_mipi0_cfg                    0       0        0        50000000    0          0     50000      Y      deviceless                      no_connection_id         
    clk_i2s                          0       0        0        50000000    0          0     50000      Y      deviceless                      no_connection_id         
    clk_uart                         0       0        0        50000000    0          0     50000      Y      deviceless                      no_connection_id         
    clk_slow_sys                     1       1        0        50000000    0          0     50000      Y      deviceless                      no_connection_id         
    pll_video_core                   1       1        0        0           0          0     50000      Y      deviceless                      no_connection_id         
       pll_video_sec                 0       0        0        0           0          0     50000      Y         deviceless                      no_connection_id         
       pll_video                     0       0        0        0           0          0     50000      Y         deviceless                      no_connection_id         
          pll_video_pri_ph           0       0        0        0           0          0     50000      Y            deviceless                      no_connection_id         
    pll_audio_core                   1       1        0        1536000001  0          0     50000      Y      deviceless                      no_connection_id         
       pll_audio_tern                0       0        0        80842106    0          0     50000      Y         deviceless                      no_connection_id         
       pll_audio_sec                 0       0        0        153600001   0          0     50000      Y         deviceless                      no_connection_id         
       pll_audio                     0       0        0        61440000    0          0     50000      Y         deviceless                      no_connection_id         
          pll_audio_pri_ph           0       0        0        30720000    0          0     50000      Y            deviceless                      no_connection_id         
    pll_sys_core                     3       3        0        1000000000  0          0     50000      Y      deviceless                      no_connection_id         
       pll_sys_sec                   1       1        0        125000000   0          0     50000      Y         deviceless                      no_connection_id         
          clk_eth                    1       1        0        125000000   0          0     50000      Y            1f00100000.ethernet             tx_clk                   
                                                                                                                    deviceless                      no_connection_id         
       pll_sys                       2       2        0        200000000   0          0     50000      Y         deviceless                      no_connection_id         
          clk_mipi1_dpi              0       0        0        200000000   0          0     50000      Y            deviceless                      no_connection_id         
          clk_mipi0_dpi              0       0        0        200000000   0          0     50000      Y            deviceless                      no_connection_id         
          clk_dpi                    0       0        0        200000000   0          0     50000      Y            deviceless                      no_connection_id         
          clk_sdio_alt_src           0       0        0        200000000   0          0     50000      Y            deviceless                      no_connection_id         
             clk_gp2                 0       0        0        200000000   0          0     50000      Y               deviceless                      no_connection_id         
          clk_sys                    4       4        0        200000000   0          0     50000      Y            1f00188000.dma                  cfgr-clk                 
                                                                                                                    1f00100000.ethernet             hclk                     
                                                                                                                    1f00100000.ethernet             pclk                     
                                                                                                                    deviceless                      no_connection_id         
          pll_sys_pri_ph             1       1        0        100000000   0          0     50000      Y            deviceless                      no_connection_id         
             clk_vec                 0       0        0        100000000   0          0     50000      Y               deviceless                      no_connection_id         
             clk_dma                 1       1        0        100000000   0          0     50000      Y               1f00188000.dma                  core-clk                 
                                                                                                                       deviceless                      no_connection_id         
 108MHz-clock                        1       1        0        108000000   0          0     50000      Y   deviceless                      no_connection_id         
    hdmi1-108MHz                     0       0        0        108000000   0          0     50000      N      107c706400.hdmi                 audio                    
                                                                                                              deviceless                      no_connection_id         
    hdmi0-108MHz                     1       1        0        108000000   0          0     50000      Y      107c701400.hdmi                 audio                    
                                                                                                              deviceless                      no_connection_id         
 27MHz-clock                         0       0        0        27000000    0          0     50000      Y   107c706400.hdmi                 cec                      
                                                                                                           107c701400.hdmi                 cec                      
                                                                                                           deviceless                      no_connection_id         
 otg                                 1       1        0        480000000   0          0     50000      Y   1000480000.usb                  otg                      
                                                                                                           deviceless                      no_connection_id         
 core                                0       0        0        50000000    0          0     50000      Y   deviceless                      no_connection_id         
 src                                 0       0        0        1000000000  0          0     50000      Y   deviceless                      no_connection_id         
 emmc2-clock                         2       2        0        200000000   0          0     50000      Y   1001100000.mmc                  no_connection_id         
                                                                                                           1000fff000.mmc                  no_connection_id         
                                                                                                           deviceless                      no_connection_id         
 uart-clock                          1       2        0        44236800    0          0     50000      Y   107d001000.serial               no_connection_id         
                                                                                                           deviceless                      no_connection_id         
 vpu-clock                           1       1        0        750000000   0          0     50000      Y   107d001000.serial               apb_pclk                 
                                                                                                           deviceless                      no_connection_id         
 osc                                 0       0        0        54000000    0          0     50000      Y   deviceless                      no_connection_id         
 clk_audio_out                       0       0        0        0           0          0     50000      Y   deviceless                      no_connection_id         
 clk_audio_in                        0       0        0        0           0          0     50000      Y   deviceless                      no_connection_id         
 clk_pwm1                            0       0        0        0           0          0     50000      Y   deviceless                      no_connection_id         
 clk_pwm0                            0       0        0        0           0          0     50000      Y   deviceless                      no_connection_id         

6by9 avatar Aug 20 '25 15:08 6by9

In 6.16 and earlier, the Pi 5 dts files have the following structure:

bcm2712-rpi-5-b.dts  // Board-specific (shared)
    bcm2712-ds.dtsi    // Downstream changes common to BCM2712-based devices
        bcm2712.dtsi    // Pure upstream file
    rp1.dtsi
    bcm2712-rpi.dtsi   // Downstream file common to all members of the Pi 5 family

The new structure is:

bcm2712-rpi-5-b.dts
    bcm2712-rpi-5-b-ovl-rp1.dts
        bcm2712.dtsi
    rp1-nexus.dtsi
        rp1-common.dtsi
    bcm2712-ds.dtsi

It lacks any clear equivalent of the last file, which is important for adding things such as RP1, the DT overrides, gpiomem declarations etc. Some of the features have been folded into bcm2712-ds.dtsi, which shouldn't contain any references to RP1, and some are in bcm2712-rpi-5-b-ovl-rp1.dts.

pelwell avatar Aug 21 '25 10:08 pelwell

Happy to restructure. The existing files were lacking in comments with regard the segregation of devices nodes.

A couple of notes:

We ought to be trying to direct upstreaming of Pi500 and CM5. As there are common blocks between those and Pi5, those want to move out of bcm2712-rpi-5-b.dts and into what mainline would normally be called bcm2712-rpi.dtsi. Is it better to create bcm2712-rpi-ds.dtsi to avoid merge conflicts with upstream gaining similar common sections?

The old files had a bundle of duplication in bcm2712-rpi-5-b.dts and bcm2712-rpi-cm5.dtsi that presumably should be factored out (eg PCIe qos settings, RP1 PCIe address layout, ethernet config, enabling rp1 blocks, etc). Sounds like they should be in bcm2712-rpi(-ds).dtsi based on your description of the split.

In 6.16, rp1.dtsi included defining a number of external clocks, /rp1_firmware and /rp1_vdd_3v3 - https://github.com/raspberrypi/linux/blob/rpi-6.16.y/arch/arm64/boot/dts/broadcom/rp1.dtsi#L1247-L1343 rp1-nexus.dtsi is included from within the &pcie2 node, therefore it can not define these external nodes. They therefore need to move, presumably bcm2712-rpi(-ds).dtsi. Or are they actually internal to RP1, and hence should be under rp1? rp1/rp1_pio then links to /rp1_firmware. /rp1_firmware links to rp1/rp1_fw_shmem and rp1/rp1_mbox. So that does imply /rp1_firmware should be under rp1. Is it not due to address mapping?

IMHO it makes more sense to directly include the downstream files from bcm2712-rpi-5-b.dts, rather than the chain of includes that we have on the other boards. bcm2712-rpi-5-b.dts, bcm2712-rpi-5-b-ovl-rp1.dts, and bcm2712.dtsi are already upstream, therefore there seems little point in breaking the include chain to insert something in the middle.

6by9 avatar Aug 21 '25 14:08 6by9

Is it better to create bcm2712-rpi-ds.dtsi to avoid merge conflicts with upstream gaining similar common sections?

We could rename bcm2712-rpi.dtsi now - it might save popcornmix a few minutes someday.

Sounds like they should be in bcm2712-rpi(-ds).dtsi based on your description of the split.

Yes, if they are effectively universal.

In 6.16, rp1.dtsi included defining a number of external clocks, /rp1_firmware and /rp1_vdd_3v3 - https://github.com/raspberrypi/linux/blob/rpi-6.16.y/arch/arm64/boot/dts/broadcom/rp1.dtsi#L1247-L1343 rp1-nexus.dtsi is included from within the &pcie2 node, therefore it can not define these external nodes.

External in what way? If you mean sitting outside the rp1 node, I don't think there's any technical reason they have to. There's no address mapping involved in the firmware - no addresses are exchanged. Clocks and regulators can be declared anywhere, even if they may need a simple-bus to get them enumerated.

IMHO it makes more sense to directly include the downstream files from bcm2712-rpi-5-b.dts, rather than the chain of includes that we have on the other boards.

Isn't that already the case? The hierarchies I included above are complete, simple as they are. The upstream additions have made it worse, but we don't need to make it worser.

pelwell avatar Aug 21 '25 15:08 pelwell

Is it better to create bcm2712-rpi-ds.dtsi to avoid merge conflicts with upstream gaining similar common sections?

We could rename bcm2712-rpi.dtsi now - it might save popcornmix a few minutes someday.

Done, and will be looking to do the equivalent as bcm2712-rpi.dtsi on mainline next week.

Sounds like they should be in bcm2712-rpi(-ds).dtsi based on your description of the split.

Yes, if they are effectively universal.

Branch updated doing that.

In 6.16, rp1.dtsi included defining a number of external clocks, /rp1_firmware and /rp1_vdd_3v3 - https://github.com/raspberrypi/linux/blob/rpi-6.16.y/arch/arm64/boot/dts/broadcom/rp1.dtsi#L1247-L1343 rp1-nexus.dtsi is included from within the &pcie2 node, therefore it can not define these external nodes.

External in what way? If you mean sitting outside the rp1 node, I don't think there's any technical reason they have to. There's no address mapping involved in the firmware - no addresses are exchanged. Clocks and regulators can be declared anywhere, even if they may need a simple-bus to get them enumerated.

Moved inside rp1 node. Seems to work.

IMHO it makes more sense to directly include the downstream files from bcm2712-rpi-5-b.dts, rather than the chain of includes that we have on the other boards.

Isn't that already the case? The hierarchies I included above are complete, simple as they are. The upstream additions have made it worse, but we don't need to make it worser.

With the new tree, yes. I was saying that I didn't want to revert to the old massive heirarchy.

Branch updated and boots Pi5 OK. Has working ethernet, USB, HDMI, serial console (although I do still have the extra entry in cmdline.txt).

One error to resolve:

[    0.776778] sram 1000120000.pcie:rp1_nexus:pci-ep-bus@1:sram@40400000: error -EINVAL: invalid resource (null)
[    0.777109] sram 1000120000.pcie:rp1_nexus:pci-ep-bus@1:sram@40400000: could not map SRAM registers
[    0.777435] sram 1000120000.pcie:rp1_nexus:pci-ep-bus@1:sram@40400000: probe with driver sram failed with error -22

Looks like address mapping issues

CM5 needs updating still to use the new files.

6by9 avatar Aug 21 '25 17:08 6by9

I didn't want to revert to the old massive heirarchy.

Show me what you consider to be the old massive hierarchy?

pelwell avatar Aug 21 '25 19:08 pelwell