Aircoookie
Add HUB75 support
Add support to drive HUB75-compatible LED matrix panels. It's a great way to get started with 2D effects in WLED at very low prices.
Summary by CodeRabbit
-
New Features
- Added support for HUB75 matrix LED panels, enabling control of these displays within the application.
- Introduced dedicated build environments for ESP32-based HUB75 matrix panel setups, including classic ESP32, Adafruit Matrix Portal ESP32-S3, and MOONHUB adapter boards.
-
Improvements
- Enhanced LED settings interface with specialized configuration, validation, and UI adjustments for HUB75 matrix types.
- Increased flexibility in pin configuration by expanding supported output pins.
- Added new pin management capabilities for HUB75, including multiple pin allocation and deallocation methods.
-
Version Update
- Updated application version.
(I'm not a WLED maintainer. My comments mean nothing.)
Wow! I'm shocked at how cleanly that slots in. Nice. I have both a Matrix S3 and a Mesmerizer (nightdriverled.com) so I may try spinning a build. What kind of frame rates are you seeing on the 2D animations? (We have a lot of the same effects in NightDriver and I've flirted with using the Faptastic HUB75 layer, too...)
Why are ESP32_FORUM_PINOUT, ARDUINO_ADAFRUIT_MATRIXPORTAL_ESP32S3, etc. compile-time flags (and not defined in a derivation of esp32dev_hub75) and not runtme-selectable flags like APA102 vs. WS2814 vs WS2812, etc.?
Given how differently -S3 and -Nothing have to be built, can you really rely on board = esp32dev? Or is this a "draw the rest of the owl" that needs to have another custom_platform_mps3.ini written that derives from an appropriate S3 with the 8MB flash, 2MB PSRAM combination? (https://github.com/platformio/platform-espressif32/blob/master/boards/adafruit_matrixportal_esp32s3.json ?) Do you need help with that, perhaps?
Would this be likely to plan nicely with the Virtual Display (chained panels - limited only by color depth and refresh rates) and UDP/DDP inputs and such?
You can see I'm pretty excited to see this as I've traveled a lot of similar paths. I'm a bit engaged in the short term, but may be able to help land this if you need extra eyes and fingers on it.
IMO, I'm OK with custom boards requiring source changes. Nobody wants a screen to manually configure a dozen plus pins.
I Do think it's preferable to be able to choose between a trinity and a matrixs3 from a menu option, just as we choose between WS2812 and SK6218 from a single binary build. So I'd vote for the #ifdefss to ve moved to a runtime selection but I'm perfectly happy to treat the collection of pins like a molecule (we COULD split it, but let's not) instead of an atom (with 16 different runtime choices). The level of granularity I'd like to see is menu option for a board with the fallback that if cracked open a soldering iron (OK, breadboard) and wired one up yourself that you're capable editing source and rebuilding.
Also, if you have PSRAM (and what new ESP32 doesn't these days?) are you really limited to one panel? The ESP32-S3s always have octal PSRAM and can DMA to and from it, lifting two major limitations from the ESP32-Nothings where PSRAM could be too slow to feep the panels and still have goofy rules where you effectively needed bounce buffers (hello ISA network cards in system with > 16MB of memory where you had to allocate in 'low memory', DMA to/from, and then copy to the real destination). Faptastic (Mr. Fap?) has some writeups on that and the ESP32-Nothings were pretty terrible for multi-panel use where the -S3's are less terrible.
Then again, if we say a single panel is all we want to mess with, this is still a super cool develoment!
On Tue, Feb 27, 2024 at 4:44 AM netmindz @.***> wrote:
@.**** commented on this pull request.
In wled00/bus_manager.cpp https://github.com/Aircoookie/WLED/pull/3777#discussion_r1504023023:
- mxconfig.gpio.b = 19;
- mxconfig.gpio.c = 5;
- mxconfig.gpio.d = 17;
- mxconfig.gpio.e = 18;
+#endif + +
- DEBUG_PRINTF("MatrixPanel_I2S_DMA config - %ux%u length: %u\n", mxconfig.mx_width, mxconfig.mx_height, mxconfig.chain_length);
- // OK, now we can create our matrix object
- display = new MatrixPanel_I2S_DMA(mxconfig);
- this->_len = (display->width() * display->height());
- pinManager.allocatePin(mxconfig.gpio.r1, true, PinOwner::HUB75);
Yeah, now I've swapped to this library for the implementation, it supports config of pins by array so I can flip to that and then use the multi call
— Reply to this email directly, view it on GitHub https://github.com/Aircoookie/WLED/pull/3777#discussion_r1504023023, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCSD3YZGDB7QMJWNR6U5ALYVW2HZAVCNFSM6AAAAABD2ZGHFOVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTSMBTGE2TQOJQHA . You are receiving this because you commented.Message ID: @.***>
Also, if you have PSRAM (and what new ESP32 doesn't these days?) are you really limited to one panel?
PSRAM is slow. Too slow for anything LED related.
That was true on ESP32-Nothing.
All the ESP32-S3's using the ESP-WROOM modules (breaking their own naming convention. Again.) like the N8R2 and N16R8 are populated with Octal PSRAM. (Boards doing their own thing like LUATOS or some of the Sparkfun/Adafruit class boards will often use QSPI and, well, they are what they are.) 120Mhz configurations are "soon" with existing hardware according to https://docs.espressif.com/projects/esp-idf/en/stable/esp32s3/api-guides/flash_psram_config.html#all-supported-modes-and-speeds , but 80Mhz is here now and is the norm on $5 boards.
Waiting for the bus arbiter to swing has a cost, so you don't want to traverse a linked list to another linked list in psram if you can help it, but block copies like a frame buffer work really well and you really can get about 80-90MB/sec in the real world doing a single-directional block move (not a RAM->RAM copy, but shoving it out a DMAC) (DDR * 1b/xfer * leading + trailing edge would say 160, but I've not seen that.)
It's not as awesome as having LPDDR5 or something and it's admittedly a house of cards to hit those rates, but it's less terrible than it used to be. We know that HUB75 can burn bandwidth at high frame rates and color depths, so there are still some tradeoffs, but don't dismiss the idea based on being burned by the ESP32-Nothings.
I think that even at QSPI (like on the LUATOS boards I have) that the wheels come off pretty easily...
On Tue, Feb 27, 2024 at 5:04 AM Blaž Kristan @.***> wrote:
Also, if you have PSRAM (and what new ESP32 doesn't these days?) are you really limited to one panel?
PSRAM is slow. Too slow for anything LED related.
— Reply to this email directly, view it on GitHub https://github.com/Aircoookie/WLED/pull/3777#issuecomment-1966303988, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCSD32TRKTCZTJ52BLX7L3YVW4TJAVCNFSM6AAAAABD2ZGHFOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRWGMYDGOJYHA . You are receiving this because you commented.Message ID: @.***>
Wow! I'm shocked at how cleanly that slots in. Nice. Thanks :)
I have both a Matrix S3 and a Mesmerizer (nightdriverled.com) so I may try spinning a build. What kind of frame rates are you seeing on the 2D animations? (We have a lot of the same effects in NightDriver and I've flirted with using the Faptastic HUB75 layer, too...)
I'm having real issues with wifi with the Matrix S3, so will be interesting to see if you see the same.
Why are ESP32_FORUM_PINOUT, ARDUINO_ADAFRUIT_MATRIXPORTAL_ESP32S3, etc. compile-time flags (and not defined in a derivation of esp32dev_hub75) and not runtme-selectable flags like APA102 vs. WS2814 vs WS2812, etc.?
If you are using the matrix portal, you can't change the pins, so no point asking the user to enter 14 pins. For other boards, just lazy ;)
Given how differently -S3 and -Nothing have to be built, can you really rely on board = esp32dev? Or is this a "draw the rest of the owl" that needs to have another custom_platform_mps3.ini written that derives from an appropriate S3 with the 8MB flash, 2MB PSRAM combination? (https://github.com/platformio/platform-espressif32/blob/master/boards/adafruit_matrixportal_esp32s3.json ?) Do you need help with that, perhaps?
The example env just shows what options you need to enable this feature. If you aren't using a regular esp32 then you need to add the extra options to the env for your board
Would this be likely to plan nicely with the Virtual Display (chained panels - limited only by color depth and refresh rates) and UDP/DDP inputs and such?
In theory yes you could use DDP then Hub75 output or mix of Hub75 and other output types
You can see I'm pretty excited to see this as I've traveled a lot of similar paths. I'm a bit engaged in the short term, but may be able to help land this if you need extra eyes and fingers on it.
Great
Also, if you have PSRAM (and what new ESP32 doesn't these days?) are you really limited to one panel?
You are not limited to single panels. At the moment it will allow you to select up to 4 panels of 32x32 in horizontal configuration. We can add other layouts later
The ESP32-S3s always have octal PSRAM and can DMA to and from it
The library being used does allow PSRAM use
Rats. The email gateway ate my response:
I finally tried to check this tree out and build it for MatrixS3. I may fumble with WLED and GitHub, but I'm not an inexperienced dev, so please stick with me.
$ gh repo clone netmindz/WLED hub75-wled Branches didn't come across, so I did $ git checkout 421c470e9b7903eed8d0f9ea6b4cff02e6050934 and confirmed the HUB75 changes were in the git log.
$ pio run -e adafruit_matrixportal_s3_hub75
npm run build [ ... ]
[email protected] build node tools/cdata.js
node:internal/modules/cjs/loader:1146 throw err; ^
Error: Cannot find module 'inliner' Require stack:
- /Users/robertlipe/src/nightdriverstrip/hub75-wled/tools/cdata.js
at Module._resolveFilename (node:internal/modules/cjs/loader:1143:15)
at Module._load (node:internal/modules/cjs/loader:984:27)
at Module.require (node:internal/modules/cjs/loader:1234:19)
at require (node:internal/modules/helpers:176:18)
at Object.
(/Users/robertlipe/src/nightdriverstrip/hub75-wled/tools/cdata.js:20:17) at Module._compile (node:internal/modules/cjs/loader:1375:14) at Module._extensions..js (node:internal/modules/cjs/loader:1434:10) at Module.load (node:internal/modules/cjs/loader:1206:32) at Module._load (node:internal/modules/cjs/loader:1022:12) at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:142:12) { code: 'MODULE_NOT_FOUND', requireStack: [ '/Users/robertlipe/src/nightdriverstrip/hub75-wled/tools/cdata.js' ] }
I don't do Node, so I'm stumped on that one. (I have built WLED directly for another S3 board from teh main repo about a month ago...)
If I restart it, it starts to chow down on the CPP code:
Compiling .pio/build/adafruit_matrixportal_s3_hub75/lib7c9/FastLED/FastLED.cpp.o
Compiling .pio/build/adafruit_matrixportal_s3_hub75/lib7c9/FastLED/hsv2rgb.cpp.o
I agree. There is an assert.h there and it's trying to include an <sdkconfig.h>, but none is in the system header path.
If I look at all the sdkconfig.h's on my entire system, I think the pattern is "projects built with idf.py, not platformio", but I'm not certain of that pattern.
If I change strategies to: $ pio run -e esp32s3dev_8MB_PSRAM_qspi I still get the failure in node.js
and the C++ compilles until it gets to wled00/wled_server.cpp:3:10: fatal error: html_ui.h: No such file or directory The only html_ui.h I have is in that aforementioned wled build directory, which makes me suspect this is a generated file. I can't help but wonder if it's generated in a step that comes after that nodejs failure.
But that same assert.h is still trying to include a sdkconfig.h that still doesn't exist, so there must be something else going on. I suspect we're getting the wrong assert.h
I see adafruit_matrixportal_esp32s3.json in .platformio/platforms/espressif32/boards so I don't think this board is too new to be supported by platformio, but it would be totally on brand for them to not print "something not found" and just blunder on with the wrong one if they did. (My relationship with PlatformIO is strained.) OTOH, it's clearly finding at JSON because it's expanding out the full name: PLATFORM: Espressif 32 (6.5.0) > Adafruit MatrixPortal ESP32-S3 HARDWARE: ESP32S3 240MHz, 320KB RAM, 8MB Flash
So I guess this is a three part question:
- What's the solution to the Error: Cannot find module 'inliner' ? 1a) Is this what's preventing html_ui.h from being built? (Assuming it's generated at all since it's no in my checkout)
- Is there something else required to build the adafruit_matrixportal_s3_hub75 target? Maybe something magic in Platformio or arduino32?
As a sanity check $pio run -e esp32dev_hub75 has the same issue with html_ui.h and with node, but does find a happy assert.h.
So I think I just have startup blues. Should this be expected to build from this basic recipe? Is this perhaps divergence from the two wled trees?
Thanx for the hand.
The simple option @robertlipe would just be to use this ready-made bin
https://github.com/netmindz/WLED/actions/runs/8056668690/artifacts/1276931005
If you are trying to build from source, you need to run
npm install
To have the right modules installed needed for inliner
Thanx.
npm install helped get through the first issue. The actual cpp code still fails.
Does pio run -e adafruit_matrixportal_s3_hub75 work for you?
pio run -e esp32dev_hub75 runs to completion.
It's definitely the board = adafruit_matrixportal_esp32s3 line that's croaking it. Changing that to match the other makes it happy, so there's something about this board configuratino tha tmakes platformio grumpy, though it's clearly recognizing it.
Will test the prebuilt binary soon...
On Mon, Mar 4, 2024 at 6:20 PM netmindz @.***> wrote:
If you are trying to build from source, you need to run
npm install
— Reply to this email directly, view it on GitHub https://github.com/Aircoookie/WLED/pull/3777#issuecomment-1977718202, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCSD3YRLQ5OLGPUJE25NNLYWUFWVAVCNFSM6AAAAABD2ZGHFOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXG4YTQMRQGI . You are receiving this because you were mentioned.Message ID: @.***>
@robertlipe - I suspect the wrong version of the platform is being used. We have the option to build on V4, which is what I use for the https://github.com/MoonModules/WLED/pull/108
Thanks @netmindz. I also have Adafruit MatrixPortal S3 board. I will try this.
The simple option @robertlipe would just be to use this ready-made bin
https://github.com/netmindz/WLED/actions/runs/8056668690/artifacts/1276931005
Should I use the following command to flash Adafruit MatrixPortal S3 board?
esptool.py write_flash 0x10000 adafruit_matrixportal_esp32s3.bin
The simple option @robertlipe would just be to use this ready-made bin
https://github.com/netmindz/WLED/actions/runs/8056668690/artifacts/1276931005
Should I use the following command to flash Adafruit MatrixPortal S3 board?
esptool.py write_flash 0x10000 adafruit_matrixportal_esp32s3.bin
Not sure about that, I just tend to goto the manual ota page of an existing WLED install and upload the bin
Not sure about that, I just tend to goto the manual ota page of an existing WLED install and upload the bin
Which firmware file did you use for WLED install before going to the manual ota page and upload your HUB75 version? https://github.com/Aircoookie/WLED/releases/tag/v0.14.1
- WLED_0.14.1_ESP32-S3_16MB_PSRAM_opi.bin
- WLED_0.14.1_ESP32-S3_8MB.bin
- WLED_0.14.1_ESP32-S3_8MB_PSRAM_opi.bin
- WLED_0.14.1_ESP32-S3_8MB_PSRAM_qspi.bin
Not sure about that, I just tend to goto the manual ota page of an existing WLED install and upload the bin
Which firmware file did you use for WLED install before going to the manual ota page and upload your HUB75 version? https://github.com/Aircoookie/WLED/releases/tag/v0.14.1
- WLED_0.14.1_ESP32-S3_16MB_PSRAM_opi.bin
- WLED_0.14.1_ESP32-S3_8MB.bin
- WLED_0.14.1_ESP32-S3_8MB_PSRAM_opi.bin
- WLED_0.14.1_ESP32-S3_8MB_PSRAM_qspi.bin
Ah, for the matrix3 I didn't actually try a regular install, just went directly to building from source. Does the web installer give a basic working setup? https://install.wled.me/
There is support for hub75 in the latest MoonModules builds, just use V4_S or V4_M builds
As for support in AirCookie, @blazoncek said he would be updating the PR to address an issue he thought was a blocker to accpeting https://github.com/Aircoookie/WLED/pull/3777#discussion_r1503852407
@netmindz I've tried to come up with a better solution regarding conditional UI elements for setting options for Hub panels. Unfortunately I was not successful as that would require redesign of whole LED settings page and bus manager logic together with available LED types. I tried that but ended efforts as there was too little gain for too much work in the end.
I can see 2 ways forward as a result of this: a) make Hub 75 panels regular option and not a conditional compile or b) not support Hub 75 at all
For a) that would probably increase code size which most users would rather see used by other usermods instead (current flash utilisation is already above 95% while we are still with ESP32 core 3.5.0, newer cores add substantial amount to this).
There is of course still possibility to keep PR as is and hide Hub panels in 99.9% of installations at the expense of few wasted bytes for never used UI elements. I cannot decide either way so I'll leave the decision to @Aircoookie. I have marked the PR with keep label so it will not become stale.
For anyone else interested please upvote this PR so we can see how much interest is there and decide upon it.
Swapping to add the options to the drop-down rather than removing is the smaller problem to solve.
As we have discussed, the bigger issue is that of the pin configuration.
The "proper" fix would be for the UI to query the bus manager to ask what the config choices should be. This would both remove the need for the UI to selectively hide/rename the config choices and also mean that adding new options like say which predefined combination of pins you are using for hub75 could come from the specific bus type class
The "proper" fix would be for the UI to query the bus manager to ask what the config choices should be. This would both remove the need for the UI to selectively hide/rename the config choices and also mean that adding new options like say which predefined combination of pins you are using for hub75 could come from the specific bus type class
Yes, that's what I was referring to. There is no such way ATM (to query) and hence needs hardcoded info in HTML files.
The other problem is also the increase of binary files themselves as mentioned which must not be underestimated as there are several enhancements in the pipeline that require quite a few more bytes.
I have started the updating of the LED preferences to be built from the BusManager and it's enabled Bus @blazoncek
https://github.com/Aircoookie/WLED/pull/4056
If you can take a look at that PR, then hopefully I can then write the second PR that handles the config values for each LED type
You can see the "clean" version of the PR, without the changes in these two PRs designed to help simplify adding new output types such as this https://github.com/Aircoookie/WLED/pull/4056 and https://github.com/Aircoookie/WLED/pull/4058
at this link - https://github.com/netmindz/WLED/compare/LED-Prefs-BusManager-config...netmindz:WLED:HUB75-AC
Are you happy that it's starting to look like a more suitable set of changes @blazoncek ? Note how it requires no change to the settings_leds.htm at the moment.
To better handle the pin config, and other settings for Hub75 we will need to change settings_leds.html to allow more to be defined in the json that just the label to be used for the BusConfig.pins entries, but I'll add those once https://github.com/Aircoookie/WLED/pull/4058 is merged so as to keep each PR clean and avoid conflicts between branches
Thank you @netmindz ! When I can, I'll join the effort.
Are you happy that it's starting to look like a more suitable set of changes @blazoncek ? Note how it requires no change to the settings_leds.htm at the moment.
Happiness has nothing to do with this but yes, I am happy. 😄
So what is the main difference between #4056 and #4058? What would be the needed order of merges?
Thank you @netmindz ! When I can, I'll join the effort.
Great 😃
Are you happy that it's starting to look like a more suitable set of changes @blazoncek ? Note how it requires no change to the settings_leds.htm at the moment.
Happiness has nothing to do with this but yes, I am happy. 😄
Great 😃
So what is the main difference between #4056 and #4058? What would be the needed order of merges?
Rather than one mega PR that does a whole load of changes, I thought would be easier to do the very low risk #4056 first, get that merged in. I can then be working on #4058 alongside this PR so we have the dynamic config we need to support all of the options we would want to see for HUB75, which would ultimately include drop-downs for things like the layout of multiple panels
I've temporarily changed the target of the merge to bus-config so we can see just my HUB75 changes, not the bus-config changes as well @blazoncek
@netmindz one thing popped up while I looked at the code (which now looks much nicer!!). How do you intend to honour ABL since you removed count field in UI?
EDIT: I think you can make it readonly and populate it's value using JS in setPinConfig().
To be honest I wasn't planning on being part of ABL.
Firstly the panels you connect might have any number of possibly LED types due to the huge variety of the number of panels out there in the market.
The second is that HUB75 works more like a CRT monitor where you are refreshing the pixels by scanning over them at high speed and relying on the persistence of vision - you never actually light up all LEDs at the same time, only two or four of the rows.
Therefore the calculation of the current draw is not possible to calculate on the same basis as addressable LEDs, any calculation used would need to be based on the width of the matrix and then half scan or quarter scan panel type