feat: Improve onyxsdk-pen integration

[zinstack625]

Updated onyxsdk dependencies

Makes stuff work on some newer devices. Don't really know why, but before that, stylus input resulted in full refreshes, as if there were no support for that altogether. This was observed both on Play Store and on local builds. May, but should not break other devices support

Reworked OnyxsdkPenArea.kt

• Removed explicit drawing Much of the stuff done in drawPreview had absolutely no effects. Whole magic seems to happen in TouchHelper. Removing that manual drawing, again, may break stuff on other devices, but most of the logic remained nonetheless. In case of breakages, should be trivial to bring that stuff back, or, if my intuition is correct, further simplify this class. Would really like to hear some feedback on that removed code if possible
• Established message passing between OnyxsdkPenArea and Dart In order to modify state of the PlatformView, message passing was required. It is used to update stroke parameters from Flutter UI toolbar to the PlatformView. I see the use as "refreshing" the creationParams. Might be useful in the future, both in of itself and as a precedent for other usecases
• Mapped used tools to those in onyxsdk-pen This in effect resulted in directly-refreshed preview looking very similar to the underlying image. Most of the tools are practically indistinguishable and are only insignificantly changed in shape (underlying image is less detailed) and, in some cases significantly in color during full refresh (seen on attached video). Exception being the pencil (seen on attached video), but it's still pretty close. Don't think it can get closer without some legally questionable voodoo magic with onyxsdk

Dart side

Most are due to preparation of tool parameters to Kotlin side, some are due to canvas being unaware of the tool being used. I tried to make significant changes as close to the actual use as possible, but may have introduced a couple of unnecessary abstractions and translations along the way. Could possibly be done better, but I'm inexperienced in Flutter and Dart. Hopefully the changes are not disruptive

Seems to be doing good on Note Air 4C, but would be good if someone could test on other devices. This does not seem to introduce any slowdowns, does not touch any significant logic and does not introduce any UI changes

20250320.webm

Great project BTW

[codecov[bot]]

Codecov Report

:x: Patch coverage is 96.15385% with 1 line in your changes missing coverage. Please review. :white_check_mark: Project coverage is 48.13%. Comparing base (5534312) to head (5911175).

Files with missing lines	Patch %	Lines
lib/components/canvas/canvas.dart	95.83%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1452      +/-   ##
==========================================
+ Coverage   47.99%   48.13%   +0.13%     
==========================================
  Files         118      118              
  Lines        8723     8746      +23     
==========================================
+ Hits         4187     4210      +23     
  Misses       4536     4536              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Fe-Ti]

Hi,

On my NoteAir 2 there is a huge lag (approx. 30s) after I lift my pen and before the actual stroke is rendered from scribble.

I tried to fix this issue and came up with this solution (same for eraser):

        override fun onBeginRawDrawing(b: Boolean, touchPoint: TouchPoint) {
            // begin of stylus data
            touchHelper.setRawDrawingRenderEnabled(true)
            reset()
        }

        override fun onEndRawDrawing(b: Boolean, touchPoint: TouchPoint) {
            // end of stylus data
            touchHelper.setRawDrawingRenderEnabled(false)
            reset()
            scheduleRefresh()
        }

With such toggling the lag disappeared and expirience became closer to proprietary note app.

Also there is an issue with releasing resources of touchHelper. For some reason the dispose() function is not called if I exit an app via navigation buttons/gestures (e.g. going to browser or launcher). So the pen continues to scribble all over the screen :)

[zinstack625]

On my NoteAir 2 there is a huge lag (approx. 30s) after I lift my pen and before the actual stroke is rendered from scribble

Regarding this lag, this should be handled in scheduleRefresh(). Setting it too low may result in display refreshing too often, which in itself may be undesirable(?). 1 sec should be reasonable, but 30 secs is definitely unintended behavior. Perhaps forcing a refresh with EpdController is better than hoping it would on itself...

Also there is an issue with releasing resources of touchHelper. For some reason the dispose() function is not called if I exit an app via navigation buttons/gestures (e.g. going to browser or launcher). So the pen continues to scribble all over the screen :)

Can reproduce, dispose() is indeed not called on app minimization, and is even skipped on app close (however view is still destroyed and this sideeffect is gone). No method from PlatformView is called on this action. I guess the best option is to figure whether canvas is shown from Dart side (is there such a callback somewhere?) and pass it to Kotlin...

override fun onMethodCall(@NonNull call: MethodCall, @NonNull result: Result) {
    if (call.method == "updateStroke") {
        val params = call.arguments<Map<String, Any>?>()
        updateStroke(params)
        result.success(null)
    } else if (call.method == "disableDraw") {
        touchHelper.setRawDrawingEnabled(false)
    } else if (call.method == "enableDraw") {
        touchHelper.setRawDrawingEnabled(true)
    } else {
        result.notImplemented()
    }
}

[Fe-Ti]

I guess the best option is to figure whether canvas is shown from Dart side (is there such a callback somewhere?) and pass it to Kotlin...

In such way overriding dispose() in _OnyxSdkPenAreaState may help

  @override
  void dispose() {
    channel.invokeMethod('disableDraw', creationParams).catchError((e) {});
  }

I'm not sure if it doesn't break anything in onyxsdk package interface.

[Iey4iej3]

I am unable to compile and test, but did you encounter #1459 before the fix?

[zinstack625]

I am unable to compile and test, but did you encounter #1459 before the fix?

Kinda, in some circumstances, not really sure, might just be my writing. If it is there, it will probably still be there, I did not mess with the code that gets the stylus data that's used after the refresh. I however can not test anything before 800e450dc6aec7a31b8acbe27229d51969dd301a, stuff just straight up does not work for me until that (I have a note air 4c, which is probably too new for version used before that)

[Iey4iej3]

Kinda, in some circumstances, not really sure, might just be my writing.

The most obvious strange thing is that, the top dot in "i" is thickened into a bullet. It does not look like this in the preview, but becomes this after a refresh.

[Iey4iej3]

Are you capable to build Saber with this patch by GitHub Action? I want to test. I try to fork and build, but there are errors.

Update: Solved the errors.

[Iey4iej3]

I am unable to compile and test, but did you encounter #1459 before the fix?

Kinda, in some circumstances, not really sure, might just be my writing. If it is there, it will probably still be there, I did not mess with the code that gets the stylus data that's used after the refresh. I however can not test anything before 800e450, stuff just straight up does not work for me until that (I have a note air 4c, which is probably too new for version used before that)

Actually, your patch seems to fix #1459. At least, it is much better. To clarify, it is not fixed by only upgrading onyxsdk as in 800e450. It is unclear why this patch fixes the issue. Thank you very much!

[Iey4iej3]

I hope that somebody could take up to improve this patch.

[jdkruzr]

hey there -- it sounds like this Onyx SDK work is more or less complete but hasn't been merged yet? or am I missing something?

[Iey4iej3]

There are some comments above on coding. I suspect that these improvements of coding could be done by AI. I do not know how to do that though.

[jdkruzr]

has anyone else noticed that when you include this stuff it breaks the Text control?

[zinstack625]

Sorry I've been out of touch, been busy with lots of stuff. I'll try looking into all the stuff soon

[zinstack625]

Okay, seems like dart code is more tolerable now. While I was at it, I also made a small "feature" in the overlay to be able to select "disabled" tool from dart, which should make some stuff like erasing and editing text a bit more tolerable.

has anyone else noticed that when you include this stuff it breaks the Text control?

With https://github.com/saber-notes/saber/pull/1452/commits/591117599307563e6708885644405f40c0510910 editing text is better. Typing on keyboard with a stylus however still initiates immediate draw, which blocks the screen refresh. At 2AM today I'm out of ideas both with how it is possible and how to fix that besides disabling the overlay altogether in those cases.

With small batch of ~~hopelessly banging the head against the table~~ experiments all I could do is break stuff. I'll try throwing some more stuff at a wall in the general area of https://github.com/zinstack625/saber/blob/main/packages/onyxsdk_pen/android/src/main/kotlin/com/example/onyxsdk_pen/OnyxsdkPenArea.kt#L125 this week and maybe something will happen...

[jdkruzr]

I've been hacking on it with Claude Code; I may have him take a crack at fixing this tonight as well. Really appreciate you diving back into this, man.

On Sun, Jul 27, 2025, 6:03 PM Антон Климанов @.***> wrote:

zinstack625 left a comment (saber-notes/saber#1452) https://github.com/saber-notes/saber/pull/1452#issuecomment-3124778713

Okay, seems like dart code is more tolerable now. While I was at it, I also made a small "feature" in the overlay to be able to select "disabled" tool from dart, which should make some stuff like erasing and editing text a bit more tolerable.

has anyone else noticed that when you include this stuff it breaks the Text control?

With 5911175 https://github.com/saber-notes/saber/commit/591117599307563e6708885644405f40c0510910 editing text is better. Typing on keyboard with a stylus however still initiates immediate draw, which blocks the screen refresh. At 2AM today I'm out of ideas both with how it is possible and how to fix that besides disabling the overlay altogether in those cases.

With small batch of hopelessly banging the head against the table experiments all I could do is break stuff. I'll try throwing some more stuff at a wall in the general area of https://github.com/zinstack625/saber/blob/main/packages/onyxsdk_pen/android/src/main/kotlin/com/example/onyxsdk_pen/OnyxsdkPenArea.kt#L125 this week and maybe something will happen...

— Reply to this email directly, view it on GitHub https://github.com/saber-notes/saber/pull/1452#issuecomment-3124778713, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEW5VXRZ4FFJQVLU7TA7Y33KVLC5AVCNFSM6AAAAABZMB7TR2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCMRUG43TQNZRGM . You are receiving this because you commented.Message ID: @.***>

[jdkruzr]

Okay, seems like dart code is more tolerable now. While I was at it, I also made a small "feature" in the overlay to be able to select "disabled" tool from dart, which should make some stuff like erasing and editing text a bit more tolerable.

has anyone else noticed that when you include this stuff it breaks the Text control?

With 5911175 editing text is better. Typing on keyboard with a stylus however still initiates immediate draw, which blocks the screen refresh. At 2AM today I'm out of ideas both with how it is possible and how to fix that besides disabling the overlay altogether in those cases.

With small batch of ~hopelessly banging the head against the table~ experiments all I could do is break stuff. I'll try throwing some more stuff at a wall in the general area of https://github.com/zinstack625/saber/blob/main/packages/onyxsdk_pen/android/src/main/kotlin/com/example/onyxsdk_pen/OnyxsdkPenArea.kt#L125 this week and maybe something will happen...

did you fix this? I just built and ran it on my Tab Ultra C Pro and it seems fine -- I can get in and out of the text tool without it crashing or losing the Onyx SDK support -- with the exception of very quick strokes sometimes resulting in big filled circle "bubbles" -- especially things like the middle stroke of a capital H. it seems to be a worse problem right after you open or create a note.

also -- can you explain why it still first draws according to how your pen writes and then after a delay seems to re-render using some other visualization library? is it not possible to just keep the first set of strokes? I feel like Flutter is doing something unwanted of its own accord here but I don't know enough yet to diagnose what it is.

I also wonder if there's maybe a good reason to have a toggle in the settings menu to turn Onyx support on and off, even just from a code path separation point of view.