2 SPI displays at the same time

[misaacson01]

Hello! I'm working on fun project in which it would be very very helpful to get 2 SPI displays working simultaneously. It's for a scientific research application (I'm happy to go into more detail!) but this could be cool to get working for plenty of other reasons (custom VR goggles anyone?) I am not expecting this to be developed at my request if there aren't any current plans to and I know I'm not the first to ask about this, but I figured it's worth checking. And if there isn't, I would really appreciate whatever help/guidance I could get in tackling this in my own fork as I'm a novice at this stuff, I apologize if there is a better place to write this that I didn't find.

Right now, I'm thinking of two different ways to accomplish this that I'd love to get feedback on:

1. Set up both hardware SPI connections and use them simultaneously, either by sending different subsets of the framebuffer to SPI0 and SPI1 or by deciding/alternating which full framebuffer is sent to which display -- I actually prefer the latter method for my application. Either way this seems hard, I supposed I'd have to duplicate a lot of stuff (especially if there are two different "diffs"?) and there's a lot I don't understand right now, but if this is what I have to do then I'll give it a go. This would be the most impressive solution I think.

2. Connect both displays to SPI0 but on different CS lines and control the two displays that way. The obvious worry is that it wouldn't be fast enough, but the specific displays I'm using are actually small and mostly circular, 16-bit color 240x210 with ~17% of the pixels cut off at the corners, which I have been using with a fast microcontroller (teensy 4.0) at 96 MHz SPI speed using the Adafruit_GFX library. So doing the back of the envelope calculation, even if I disable the beautiful frame diffing for development simplicity and send every pixel every time, I think I could get close to 80 fps (the max refresh rate of the display I believe) with 2 displays simultaneously on a single SPI connection? And if I ignore the diffing, and initialize both displays at the start, might this be as simple as just changing the value of SPI0_CS after deciding where to send a frame?

I would love to talk more about this to anyone who is willing and interested! One way or another I will get this figured out...

[juj]

2\. Connect both displays to SPI0 but on different CS lines and control the two displays that way.

1. by sending different subsets of the framebuffer

That would probably be the best approach, if your source application can generate content that way. E.g. set up the framebuffer size in /boot/config.txt to work with 240x420 framebuffer, and when a frame is snapshotted, route the top 240x210 part to one display, and bottom 240x210 part to the other display, flipping CS when communicating to each display. Diffing would be done separately for the two parts.

If both displays update at equal content frequency, then frame rate is essentially halved. You can benchmark what you'd get by setting up one display, then building with https://github.com/juj/fbcp-ili9341/blob/662e8db76ba16d86cf6fd09d85240adc19e62735/gpu.cpp#L21 uncommented. The frame rate that you'd obtain there would effectively be halved.

There is currently no support for multiple displays, so you would have to develop that yourself. For that you'll need to familiarize yourself with the BCM2835 hardware SPI controller: https://www.raspberrypi.org/app/uploads/2012/02/BCM2835-ARM-Peripherals.pdf . See pdf pages 148-159.

In particular the SPI0 CS register controls the active peripheral line. Changing CS 1:0 bits from 00 (Chip Select 0) to 10 (Chip Select 1) at wherever spi->cs = is assigned to (e.g. https://github.com/juj/fbcp-ili9341/blob/662e8db76ba16d86cf6fd09d85240adc19e62735/spi.cpp#L293 for polled transfers, and https://github.com/juj/fbcp-ili9341/blob/662e8db76ba16d86cf6fd09d85240adc19e62735/dma.cpp#L533 for DMA transfers, and a number of other places (see where DISPLAY_SPI_DRIVE_SETTINGS is being referenced) will flip from SPI0 CE0 (GPIO 8/BCM 24) active to SPI0 CE1 (GPIO 7/BCM 26) active.

Then probably the task queue would receive in addition to the tasks, a bit that records whether that task should be aimed towards the CE0 or CE1 device.

Hopefully that helps getting going, good luck with the project!

[misaacson01]

Thank you so much for the awesome notes and fast reply! I will try the method you suggest to benchmark with one display to make sure the single SPI would be good enough. And just to make sure I better understand this solution (this will show my ignorance...):

• both displays get initialized, I supposed by looping through the display init sections and maybe including another argument into all the SPI task functions (e.g. in spi.h) the bit that determines CE0 or CE1.
• The full 420x240 frame (which I believe I can generate content that way but haven't actually done yet) gets snapshotted, nothing needs to be changed about that other than setting up the size you mentioned in config.
• I would guess that the full framebuffer gets diffed as a whole, otherwise all the sizes will have to be corrected? Or should I spilt the framebuffer into top and bottom halves before the diff, perhaps in fbcp-ili9341.cpp? I'm not sure what all that entails but I'll keep studying.
• Then when it comes time to do the SPI tasks to send the framebuffer halves to the two displays, I suppose I would duplicate or loop through that in fbcp-ili9341.cpp, though still not sure yet how much or how little of it I have to do twice. But whatever I do have to do twice, like you said, everywhere a SPI task is called I could include that chip select bit, and inside the tasks in spi.cpp and dma.cpp I could use that bit something like: spi->cs = BCM2835_SPI0_CS_TA | BCM2835_SPI0_CS_CLEAR_TX | DISPLAY_SPI_DRIVE_SETTINGS | **CS_TARGET_BIT**; since it looks like the CS register just needs bit 0 to be switched since I'll only connect to CE0 and CE1 (00=CE0, 01=CE1, 10=CE2)?

[juj]

• both displays get initialized, I supposed by looping through the display init sections and maybe including another argument into all the SPI task functions (e.g. in spi.h) the bit that determines CE0 or CE1.

Sounds about right. Probably the tasks in the queue should receive a bit whether the task is for CE0 or CE1. That way operation can be queued for both displays without having to wait for one to flush the other.

• I would guess that the full framebuffer gets diffed as a whole, otherwise all the sizes will have to be corrected? Or should I spilt the framebuffer into top and bottom halves before the diff, perhaps in fbcp-ili9341.cpp? I'm not sure what all that entails but I'll keep studying.

The halves should be diffed separately, otherwise a number of issues will occur. (coordinates are wrong, diff can straddle the split line etc.)

• But whatever I do have to do twice, like you said, everywhere a SPI task is called I could include that chip select bit, and inside the tasks in spi.cpp and dma.cpp I could use that bit something like: spi->cs = BCM2835_SPI0_CS_TA | BCM2835_SPI0_CS_CLEAR_TX | DISPLAY_SPI_DRIVE_SETTINGS | **CS_TARGET_BIT**; since it looks like the CS register just needs bit 0 to be switched since I'll only connect to CE0 and CE1

Yeah, that sounds right.

[misaacson01]

The halves should be diffed separately, otherwise a number of issues will occur. (coordinates are wrong, diff can straddle the split line etc.)

I see, that makes sense. Thank you for the head start!

[egemenertugrul]

Hey there - I'm working on a similar VR project too!

The tips given above are quite informative, thanks @juj. I was wondering if you had any success with the development? @misaacson01

Thanks in advance.

[misaacson01]

@egemenertugrul Nice timing, I was just revisiting a difficulty I had with this project this week! Once I've finished going through this again I'll write something up and let you know how it goes, probably Thursday or Friday.

[misaacson01]

@egemenertugrul I have picked up this project again and made some progress. I can successfully connect 2 displays on the CE0 and CE1 chip selects, I can initialize them both by looping through the init sequences and adding a CS_TARGET to all the spi transfer stuff, and I can send frames to both displays by switching the bit in the main program loop. I'll update my repo once I figure a little more out.

Using functions like SPI_TRANSFER to send things to one display or the other is super easy, though the trickier thing to figure out has been to switch the chip select on a task-by-task basis (since tasks can get queued). After trying different things, what I have in mind now (which sometimes works?) is by adding the chip select target as a member of the SPITask struct, so whenever the SPI thread grabs a task, it can check (e.g. task->csTarget) to see who it's for. A problem I'm running into now is that this new csTarget member seems unstable in SPITask? When I check it's value, it's kinda all over the place, even though I'm only ever setting it to be 0 or 1. I assume the issue is that the SPITask addresses have to be structured a very specific way, but I'm still a novice at this kind of programming so I don't understand it yet. I guess this is a good time to check in with @juj to see if this is the right track? I appreciate any new advice!

[leoshmu]

@misaacson01 your progress is super exciting! In the original readme @juj stated that you could connect HDMI and SPI TFT at the same time but at baseline they would show the same thing, because "at the moment fbcp-ili9341 has been developed to only display the contents of the main DispmanX GPU framebuffer over to the SPI display". So did you solve this as well?

[misaacson01]

@leoshmu No I didn't solve/change that, it still can only show what's in the main DispmanX GPU framebuffer, but with the idea being (and what I'm working on right now) that the 2 SPI displays will show different subsets of that full framebuffer.

[smoscar]

I'm interested in contributing to this.

I have been testing with 2 waveshare32b screens but haven't been able to make this work on different CS chips. When I connect them to the same chips it sometimes work though. What driver do your screens run on?

[misaacson01]

@smoscar I had to make a new driver for my screens: gc9307 (which was pretty similar to st7735), but I think connecting two of these to the same CS worked pretty much right away. I wondered how the SPI signal quality would be affected by adding a 2nd board to the same connections, so I kept my wires pretty short.

[leoshmu]

@leoshmu No I didn't solve/change that, it still can only show what's in the main DispmanX GPU framebuffer, but with the idea being (and what I'm working on right now) that the 2 SPI displays will show different subsets of that full framebuffer.

Thanks. One thing I've wondered is what happens if the frame buffer input is a different dimension than the spi screen. For example, if the screen is 720x480 and you feed a 1440x480 feed, will it clip half of it? Is there any tricks you could play to the same oversized frame buffer to both screens, but one screen clips it 'left-justified' and the other clips is 'right-justified'?

[misaacson01]

Thanks. One thing I've wondered is what happens if the frame buffer input is a different dimension than the spi screen. For example, if the screen is 720x480 and you feed a 1440x480 feed, will it clip half of it? Is there any tricks you could play to the same oversized frame buffer to both screens, but one screen clips it 'left-justified' and the other clips is 'right-justified'?

Yep, that's exactly what I'm thinking. Right now, I'm using the setting for DISPLAY_CROPPED_INSTEAD_OF_SCALING which crops out a little piece of the center of the framebuffer and sends that to the display, with the intent of getting 2 cropped regions side-by-side (haven't done that yet). And then keeping each cropped section in its own framebuffer so that I can keep 2 copies of each (previous frame and current frame) so I can use all the diffing functions on each thing separately and send spans separately. I don't really know what to think about the possibility of frame interlacing (would I want the possibility of 1 display interlaced and the other not?), maybe I'll just keep track of interlacing for each frame separately too.

[smoscar]

@smoscar I had to make a new driver for my screens: gc9307 (which was pretty similar to st7735), but I think connecting two of these to the same CS worked pretty much right away. I wondered how the SPI signal quality would be affected by adding a 2nd board to the same connections, so I kept my wires pretty short.

Interesting. I did start noticing that problem a lot more when I had my scope spying on the signals. Do your screens require CS1 or can you connect the second display to SPI0_C1?

[misaacson01]

Interesting. I did start noticing that problem a lot more when I had my scope spying on the signals. Do your screens require CS1 or can you connect the second display to SPI0_C1?

Definitely both screens have to be on SPI0. They both work when they're on CS0. And they mostly work when they're each on a different CS (SPI0 CS0 and SPI0 CS1) when I set the CS register to the correct chip select, though I get some problems sometimes that I don't understand, but there are maby different places where the chip select/SPI0 register has to be set so I could have missed some.

[leoshmu]

Thanks. One thing I've wondered is what happens if the frame buffer input is a different dimension than the spi screen. For example, if the screen is 720x480 and you feed a 1440x480 feed, will it clip half of it? Is there any tricks you could play to the same oversized frame buffer to both screens, but one screen clips it 'left-justified' and the other clips is 'right-justified'?

Yep, that's exactly what I'm thinking. Right now, I'm using the setting for DISPLAY_CROPPED_INSTEAD_OF_SCALING which crops out a little piece of the center of the framebuffer and sends that to the display, with the intent of getting 2 cropped regions side-by-side (haven't done that yet). And then keeping each cropped section in its own framebuffer so that I can keep 2 copies of each (previous frame and current frame) so I can use all the diffing functions on each thing separately and send spans separately. I don't really know what to think about the possibility of frame interlacing (would I want the possibility of 1 display interlaced and the other not?), maybe I'll just keep track of interlacing for each frame separately too.

Had an idea for a different approach to getting 2 different streams to 2 screens. What if each SPI screen was connected to a PiZero, and you connected each PiZero to a master Pi4 or CM4 module? Is there a way to send frames to each connected PiZero frame buffer? Maybe a master/slave setup of some sort to achieve this?

[misaacson01]

Had an idea for a different approach to getting 2 different streams to 2 screens. What if each SPI screen was connected to a PiZero, and you connected each PiZero to a master Pi4 or CM4 module? Is there a way to send frames to each connected PiZero frame buffer? Maybe a master/slave setup of some sort to achieve this?

I don't know of any good way to do that. The only way I can think of would be SPI or IIC, but IIC would be way too slow and SPI is basically the same thing we're trying to do here, except significantly added latency with an additional PiZero in the middle. Maybe the most hacky route would be to have two raspberry Pis, each connected to a single display, and have them both generate the same frame but send different subsets to their respective screen. But I'm still going to try the solution of just using one Pi4 and sending different subsets to two displays on the same SPI, using chip-selects to switch between them.

[misaacson01]

@leoshmu @smoscar @egemenertugrul and for anyone who might be interested in a follow-up: I did basically get this working on 2 displays. Some diffing options I haven't figured out yet, but currently, I can display different subsets of the framebuffer onto 2 SPI displays (in my case, a new circular display, GC9307). The changes I made to the code are basically what was planned from the beginning: the gpu frame size is set to double the height of the actual displays so that the top half would go to one display and the bottom to the other. In the main program loop, after a new frame is grabbed from the GPU (and before diffing), I start a loop through the code twice, switching a chip-select bit for each. Diffing happens on each half separately, with a row downward shift happening for the 2nd one. At least where the framebuffer is concerned, the 2nd pass is shifted downward, but where the SPI display is concerned, nothing is shifted on the 2nd pass (e.g. y cursor stays at 0 even though the first framebuffer row is halfway down). The code is in my fork here. I think it's missing an important update but I'll push it soon. This will not work right off the bat for other displays, I only edited the driver file for mine and I hardcoded all of my specific build options. But it wouldn't be hard to get it working on other displays.

[egemenertugrul]

This is fantastic news - thank you for sharing @misaacson01! I switched to using a single display for my project, but if I ever were to use two, this is where I'll come back to.

[QRT8]

Hi,

Didn't know where to post this so I'm doing it here. I want to make a new driver to make this work for the ST7789, and I'd appreciate some help

@smoscar I had to make a new driver for my screens: gc9307 (which was pretty similar to st7735), but I think connecting two of these to the same CS worked pretty much right away. I wondered how the SPI signal quality would be affected by adding a 2nd board to the same connections, so I kept my wires pretty short.

You said the ST9307 driver is similar to the ST7735 and ST7789 displays use the same driver. So is it easy to port your driver to the ST7789? How could I do that? (Hope I'm not asking for too much)

Thanks in advance!

[misaacson01]

@QRT8 Making a new driver is pretty simple once you get the format, especially if it's similar to one already made. Though are you trying to just get this original juj repo working on the ST7789 (only using 1 display) or are you trying to have 2 different ST7789 displays show different things? The former should be very simple, the latter a bit less simple but still doable.

[QRT8]

@misaacson01 Thank you so much for the fast reply. I basically want to show different things on two displays (like you did) for custom AR goggles. So I guess I can just write a new st7735r.cpp (probably based on yours for the gc9307) to do that. Is there anything else I have to edit in your repo besides changing the resolution?

[misaacson01]

@QRT8 There's a few changes you'll need to make from my repo. Which driver do you need to use exactly? Is it ST7789(VW) or ST7735(R or S)?

Here are some things I know you need to edit:

1. If you're making a new driver entirely, Add it as a build option in cmakelists.txt (and in display.h). If you're going to edit an existing driver, make sure you specify it when building, and you may want to remove the default to GC9307 (lines 283-284)
2. At the top of config.h, set the D/C and Reset pins to whatever you'll be connecting to. Also change the framerate and clock divisor to something that makes sense for you.
3. Create/edit the driver files (e.g. st7789.cpp and st7789.h) to match what they look like in the gc9307 driver files.

I think that's it? Let me know how it goes!

[QRT8]

@misaacson01 Thanks for your help! I changed the st7735r driver yesterday, as it is the one that works with the st7789 (you can see the changes in my repo). I tested it today, but only one display shows a portion of the framebuffer with different colors, while the other one remains black. What Cmake options did you use? Do I have to turn off the diffing manually? And should I change the hdmi resolution to 240*480? or does it also work with higher resolutions?

Thats the display that works (should show white instead of green) and the one that doesnt work.

[misaacson01]

@QRT8 I didn't use any cmake options that isn't hard coded, so you should be good there. But I see that my repo is still trying to use diffing (which I was testing but was still not quite working). So in config.h, uncomment the alltasksshoulddma and updateframeswithoutdiffing. And maybe it's worth to look out for any newer change I might have made in another of my repositories, which is a more recent fork: https://github.com/sn-lab/mouseVRheadset/tree/main/fbcp-ili9341