There's no LED support

[fredizzimo]

I didn't solder LEDs to the keyboard myself so I'm unable to test and implement this feature myself. It shouldn't be hard to do, using the visualization framework.

If you either want to test or implement this, please contact me.

[kejadlen]

I'd be happy to test this out. Would be interested in implementing it too, but not sure at all where to start.

[tthayer]

I'd be willing to test some things out too. Just need some guidance.

[fredizzimo]

Hi,

Sorry for not replying earlier. I have been and still am on a trip, so I haven't had time to answer and think about this.

But today I had some time in the hotel room to look more into it, It doesn't seem to be as easy as I first thought, but it shouldn't be too hard either. I think the design that makes most sense would be to expose the LEDs as gray scale display that can be controlled the same way as LCD. Each pixel would represent a led, and the color would represent the brightness.

The advantage of doing it this way, rather than something custom would be that then you could use the drawing routines of uGFX, including font rendering. Another advantage is that the same abstraction would work for keyboards with RGB leds as well.

In order to do this, a new uGFX display driver would need to be made, just like the LCD driver (in the drivers folder of the main repository). This driver would have to use the I2C functionality of the Chibios to communicate with the LCD controller according to the specs. http://www.issi.com/WW/pdf/31FL3731C.pdf.

Some tweaking of the makefiles and the chconf is needed, to enable the build of the display, and to enable multi-display support (which works slightly differently than for using just a single display) for uGFX.

A boardfile would also need to be made (also in the drivers folder), that enables the right input and output modes for the pins, to support the I2C communication

Then this would need to be integrated into the visualizer library like the LCD.

I think I can get something for you to test quite soon after I'm back. But I would of course be very happy if the instructions here are enough to get you started in actually implementing itself.

[fredizzimo]

Hi @kejadlen, @tthayer.

I have now implemented the first version of LED support. But I have no idea if it actually works. The only thing I know is that it doesn't crash, and the rest seems to work OK.

So it would be great if you could test it out.

I have submitted my changes in the led branch, so to take it into use you have to do this

git fetch origin
git checkout -b led origin/led
git submodule sync --recursive
git submodule update --init --recursive

Then make and flash both halves. When the keyboard starts you should see a looping test. It would be great if you could take a video or something, so I can see that it works properly, but it's ok if you just verify it yourselves too.

The test should go like this

1. Fade in all LEDs to full brighness in 1 second
2. Show all leds for 1 seond
3. Fade out all LEDs so that they are completely shut down in 1 second
4. Start to fade in a gradient pattern, the outside should be the brightest, and the middle the lightest, in one second
5. Move the pattern from outside to inside for 3 seconds
6. Crossfade the pattern to another one that goes from top to bottom in one second
7. Move the pattern from top to bottom in 1 second
8. Crossfade to a pattern that goes from inside to outside in 1 second
9. Move the pattern from inside to outside for 3 seconds
10. Crossfade to a pattern that goes from to to bottom in 1 second
11. Move the pattern from to to bottom for 3 seconds
12. Fade out all leds in 1 second
13. Start over

You should report anything that isn't as you think it should be. Especially if some or all LEDs are not working, or if the patterns look wrong.

The pattern implementation is in tmk_visualizer/led_test.c And the actual low level communication mostly in drivers/gdisp/IS31FL3731C

[tthayer]

Nothing is happening with the LEDs. The display backlight on the right-hand half cuts out after a minute and doesn't come back on until the keyboard is replugged in.

[fredizzimo]

Thanks for the report. Guess I have to re-check all the code.

But the fact that the backlight cuts off is very strange. It doesn't happen on mine with no leds installed.

Does the keyboard still work if the backlight is off? And does the LCD go blank too?

[tthayer]

Thanks for working on this! The keyboard continues to function just fine and the LCD continues to cycle through the layers display. 

Sent from Outlook for iPhone

_____________________________

From: fredizzimo [email protected] Sent: Sunday, April 24, 2016 12:32 Subject: Re: [fredizzimo/infinity_ergodox] There's no LED support (#5) To: fredizzimo/infinity_ergodox [email protected] Cc: Tony Thayer [email protected], Mention [email protected]

Thanks for the report. Guess I have to re-check all the code.

But the fact that the backlight cuts off is very strange. It doesn't happen on mine with no leds installed.

Does the keyboard still work if the backlight is off? And does the LCD go blank too?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

[dervos]

Perhaps as a 'hello world' test you could hook one led port up to a volt meter. It would allow for easier initial testing and you'd be able to verify the test results of others.

[fredizzimo]

@tthayer, I found an issue, there was an off by one error in the code, which had been left by a refactoring I did.

The code was taking care of setting which LEDs are enabled to the hardware. The controller has two blocks, and only one of them is mapped to the keyboard LEDs. And of course the byte layout for those are interleaved, which means that no LEDs would get mapped. So that would explain why nothing shows up.

But this does not explain why the LCD backlight stops working. Although both are LEDs they have nothing to do with each other, and controlled by entirely different controllers. A memory corruption of some sort could be possible, but I doubt it, especially since most values should be re-written when you switch layers. I also think that this would happen on my board without LEDs as they should not affect the software at all.

Some kind of USB power issue could also be possible, if it detects that there's not enough power and shuts the LEDs off. But that also sounds very unlikely since the LEDs were not working and they shouldn't draw any power.

So for now, the most likely cause sounds like there might have been some kind of inference because of the incorrect mapping. So I suggest that you try my fix out.

@dervos Thanks for the suggestion, I guess a multimeter would show at least something, even if the LEDs are PWM controlled. I will try it, if this fix isn't enough, and there are other bugs. I don't want to open the keyboard if it's not necessary, as the acryllic is very hard to align right, at least on my keyboard. I have the first version, where they didn't fix the space between the thumb section, so I have less than one mm of space there.

[tthayer]

@fredizzimo The LEDs still don't power on but the right-hand LCD backlight stays lit now.

[fredizzimo]

@tthayer, thanks for the report, although it's completely the opposite from what I expected.

I will continue reviewing my own code, and test with a mult-meter before I ask you again.

[tthayer]

@fredizzimo No problem. I wish I was able to provide better feedback!

[fredizzimo]

@tthayer, I just pushed a new version, that forces all the LEds on directly from the driver, bypassing all visualization logic.

I get some readings when testing with the multimeter, but it varies from LED to led for some reason. Maybe you could test this to see if some of them light up?

[kejadlen]

@fredizzimo Unfortunately, I'm away from my Infinity until Thursday, but looking forward to helping out with testing and debugging once I get back.

[tthayer]

@fredizzimo That worked! They're all lit now. The right side now feels unresponsive and kind of slow though. Keys on the right side are also 'sticking'.

[fredizzimo]

@tthayer, Great. Do they all have the same brightness?

The unresponsiveness probably comes from the fact that the animation is still running in the background, and the same problem actually happens for the LCD screen too. I will have to profile and optimize some things.

@kejadlen, Don't worry, I have the help I need for now.

I will check if I can find the reason why it doesn't work through the visualizer. Probably not for too long though, as I have to go to bed quite soon.

[tthayer]

Yes, it appears they all have the same brightness.

[fredizzimo]

Hi @tthayer, @kejadlen.

I have fixed a couple of issues, and I think that the LED animation should be working now. Could you test it, please?

The brightness probably don't change as linearly as it should since I didn't apply any gamma calculations, as described in http://www.issi.com/WW/pdf/31FL3731C.pdf, read the "gamma correction" section. I'm not sure if I should apply that correction on the user side in the visualization code or, as a helper function that can be called, on the driver side, done automatically. Doing it on the user side would be more flexible, but I'm not sure if that kind of flexibility is needed. What do you think?

[kwkroll32]

@fredizzimo Do you still need help testing this? I just flashed the LED branch and it seems to be running the light test loop.

[fredizzimo]

@kwkroll32, thanks. Yes, @tthayer and @kejadlen haven't had time to test the latest version yet.

It would be nice if you could verify that everything is done in the correct order, like I described above. And that all LEDs light up exactly as described.

I'm also interested in the linearity of the brightness, since there's no gamma correction as described above. How bad is it?

Once everything is confirmed working, I guess I will disable the demo loop, and maybe replace it with a fixed brightness settings, or even all LEDs off. You should then be able to customize it the same way as the LCD screen. I will also merge it to the master at that point.

[tthayer]

I'll try it out this weekend.

[kwkroll32]

I uploaded a short video of the LED behavior. Hopefully you can tolerate the poor quality. https://youtu.be/-M3CBAGVVEY

[fredizzimo]

@kwkroll32, thanks for the video.

It looks like gamma correction is definitely needed. And there also seems to be some bugs, for example at around 0:08, it looks like half of the keyboard stays fully lit, while the other half is off.

I will look at it maybe later today or tomorrow, but if some of you want to have a look too, the bug should be somewhere in the tmk_visualizer/led_test.c file.

[kejadlen]

@fredizzimo Having trouble updating ugfx:

fatal: reference is not a tree: e221a690616e20f87e0b0088baffdbd5427be862
Unable to checkout 'e221a690616e20f87e0b0088baffdbd5427be862' in submodule path 'tmk_visualizer/ugfx'
Failed to recurse into submodule path 'tmk_visualizer'

Edit: Nevermind, I figured out the problem - git was still using Tectu instead of fredizzimo for the remote.

[bremoran]

Once my LEDs arrive, I'm happy to help out with testing this feature out.

[fredizzimo]

Hi All,

I have hopefully fixed the gradient code. I don't know what I was thinking when I wrote the original function, it seemed like I had only done it halfway.

The effect should be much better now, with this fix applied. But to make sure it would be nice if someone could test it.

I still think the gamma correction is needed though, so I will implement that next. It will be done inside the driver so it should be completely transparent to the user.

[kwkroll32]

That looks a lot better; new video below. I had a hard time discerning what it was supposed to do last time. The only thing I'm not sure about is whether the horizontal crossfade is supposed to start in the middle column. You can tell that it starts one away from the middle column in both the left-->right and the right-->left fade. https://youtu.be/lGJzUa6dBUE

[fredizzimo]

Thanks, that really confirms that brighness correction is needed.

I also think that there might be some problems with the top row, it doesn't look like it's having the same brightness as the other rows from the video.

Also the thumb cluster, seems to be acting strangely, the outer keys seems to be quite bright. And it gradient is supposed to go down below the main keyboard rows to the two thumb buttons, because I represent the keyboard as a 7x7 bitmap, where the two upper thumb keys are part of the bottom main keyboard row.

I thought that this gradient effect would be enough to diagnose that all LED works individually correctly, and it might be, when the brightness correction works. But I still think I have to add one step that lights up each led, one by one, for easier diagnosing.

[kwkroll32]

I've been reading the code and documentation to figure out how this works [having written lots of Python, but never working with C or hardware controllers]. One thing I noticed is that commenting out portions of the animation test in led_test.c can break the entire animation altogether. Do you know what that's about? I also commented out the corresponding lines from led_test.h, and reduced the number of frames to match the number of frame_functions.

In other words, how might I limit your animation test to [keyframe_fade_in_all_leds, keyframe_no_operation, keyframe_fade_out_all_leds]? I think that having these 3 as a sanity check, plus 1 complicated test like the crossfade, might make it easier for me to troubleshoot.

[fredizzimo]

Hi @kwkroll32,

I'm not very happy with the ease of use of the code myself either. I'm a C++ programmer, with very little experience in C, so I'm not very good at making good C interfaces. There's also special requirements for hardware controllers like this, like avoiding allocations, which complicates things further.

Furthermore I want to extend it with more features in the future, like being able to play animations on top of each other, a bit like layers. That would be useful for things like adjusting the brightness, while you are adjusting the brightness value would be showing, and when it's done, the display (LCD, LEDs and backlight) would resume to what it was showing before.

Therefore I have been thinking of doing a C++ interface, which would be easier to use, with less chances of making mistakes that the compiler can't detect. Originally I used C for the Visualizer because the TMK firmware is otherwise written in C, and I was also hoping that this would become a more official part of it. However that's not the case, so I could use C++ instead.

The documentation might also be lacking a bit at the moment, I will try to improve that once we have figured out what your problem is.

That said, you should easily be able to remove keyframes from the example code. You should only need to comment out or remove the frames you don't need, and change the keyframe count. But remember to always also update the list of frame lengths to match. You don't need to remove the code that actually defines those frames.

So the simple case you are talking about would look like this

keyframe_animation_t led_test_animation = {
    .num_frames = 3,
    .loop = true,
    .frame_lengths = {
        MS2ST(1000), // fade in
        MS2ST(1000), // no op (leds on)
        MS2ST(1000), // fade out
    },
    .frame_functions = {
        keyframe_fade_in_all_leds,
        keyframe_no_operation,
        keyframe_fade_out_all_leds,
    },
};

Even better, you could define your own animation in visualizer_user.c, just copy the animation example struct, but give it another name. And then start that animation instead from the function initialize_user_visualizer. Or you could of course bind different animations to different layers, and start them from the update_user_visualizer_state

If you do it that way, you can leave the led_test functions completely unchanged.

• The crossfade functions are a bit complicated, since they will use the previous and next frame to do their work. It should smoothly change the LED brightness from the previous frame to the next.
• The mirror tests are there because, it's likely that the user would want to mirror the display, when writing code that targets the right hand display. That's especially true if you display a picture or some text.
• The no operation frame, does nothing as the name says, so it will leave the leds in the exact same state as the previous frame left them.
• The gradient functions, at least when they work correctly will show a moving gradient pattern, or in other words leds with decreasing brightness. The gradient has the width or height of the half keyboard, so if you fad from left, it should start with full brightness on the left, the next column should be a little bit less bright, and the middle almost completely dark. Then the brightness will start to increase again, so that the pattern completes around the keyboard half. The pattern is a little bit hard to see at the moment because of the non-linearity of the eyes, which I'm making compensation for (brightness correction, or what I wrongly called gamma correction before).

You can of course also write your own keyframes, since these ones are more of a demo nature, than useful on a real keyboard. For that you probably have to check out the ugfx documentation

It also probably helps to increase the keyframe lengths, if you want to see what it's doing a bit better. I had to record the video with VLC, and then step frame by frame, to analyse it.

If this doesn't help you, you can post some code, and I will hopefully be able to tell you what the problem is.