engine icon indicating copy to clipboard operation
engine copied to clipboard

Add EGL Surface backing store

Open ardera opened this issue 1 year ago • 15 comments

Allows using an EGL surface as a flutter backing store. Way more convenient for GBM than hacking gbm bo's into GL FBOs.

This resolves https://github.com/flutter/flutter/issues/58363

Currently, the embedder API assumes that the compositor (if it exists) will let flutter render into FBOs or Textures and then composite the whole thing onto the actual (EGL) window surface. I think this assumption is also documented a bit in https://github.com/flutter/flutter/issues/38466

However, in my case, I want let the hardware do the composition (using the linux KMS API), and render each flutter layer into it's own EGL surface.

It's possible to hack around this by creating your own GBM BOs, importing those as EGL images, then importing those as GL Render Buffers and attaching those to GL FBOs and that works (tested it). However, that's basically reimplementing 50% of the whole GBM/EGL "window" system integration for no reason.

This PR adds:

  1. To the embedder API:
    • a new kind of OpenGL Backing store: FlutterOpenGLSurface
      • consisting of just a make_current and destruction callback (plus userdata)
      • the make_current callback should make the target surface current, i.e. eglMakeCurrent(..., surf, surf)
      • will be called by the engine before rendering onto the backing store
  2. Some wiring to call make_current before rendering into the backing store

TODO:

  • [ ] Actually handle the return value of the make current callback
  • [ ] Tests

Pre-launch Checklist

  • [x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • [x] I read the Tree Hygiene wiki page, which explains my responsibilities.
  • [x] I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
  • [x] I listed at least one issue that this PR fixes in the description above.
  • [ ] I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See testing the engine for instructions on writing and running engine tests.
  • [x] I updated/added relevant documentation (doc comments with ///).
  • [x] I signed the CLA.
  • [ ] All existing and new tests are passing.

ardera avatar Jul 14 '23 15:07 ardera

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact Hixie on the #hackers channel in Chat (don't just cc him here, he won't see it! He's on Discord!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

flutter-dashboard[bot] avatar Jul 14 '23 15:07 flutter-dashboard[bot]

I think this direction is sound if you want to wrap up the remaining TODO items.

chinmaygarde avatar Jul 20 '23 20:07 chinmaygarde

Is this good for a review? This is still marked as WIP so not sure if you are still tinkering on it.

chinmaygarde avatar Jul 27 '23 20:07 chinmaygarde

@chinmaygarde The implementation is done and won't change anymore (unless we find any flaws). Just doesn't have any tests yet

Btw, I noticed adding new fields to any of the specific, versioned backing store variants (FlutterOpenGLSurface, FlutterMetalTexture) technically violates the ABI stability rules, right? Because they're (indirectly) embedded into FlutterBackingStore by value. The only reason it doesn't matter is because there's no member after the embedded structs in FlutterBackingStore, of which the offset could change incompatibly

ardera avatar Jul 28 '23 11:07 ardera

Are you planning on adding tests or is this good for review?

Re. the ABI stuff, yeah we are getting away with it because we haven't anything to the end of the FlutterBackingStore struct (and can't).

chinmaygarde avatar Aug 10 '23 20:08 chinmaygarde

I'm currently on vacation, so next week :+1:

ardera avatar Aug 14 '23 09:08 ardera

Take your time! Just putting the WIP tag so triagers don't bother you.

chinmaygarde avatar Aug 17 '23 20:08 chinmaygarde

This pull request executed golden file tests, but it has not been updated in a while (20+ days). Test results from Gold expire after as many days, so this pull request will need to be updated with a fresh commit in order to get results from Gold.

flutter-dashboard[bot] avatar Sep 08 '23 10:09 flutter-dashboard[bot]

@ardera is this still work in progress?

cbracken avatar Oct 21 '23 01:10 cbracken

@cbracken yep it is

reworking the testing infrastructure to account for multiple EGL surfaces was a bit more work than I thought originally, so just didn't get to it now

ardera avatar Nov 01 '23 13:11 ardera

Hey @ardera, are you still working on this?

jonahwilliams avatar Nov 30 '23 21:11 jonahwilliams

@jonahwilliams Sorry, here it is. The tests mostly assume there's only a single OpenGL surface, so I kinda had to fight against the test framework a bit lol. It's ready for review now.

ardera avatar Jan 08 '24 14:01 ardera

@ardera this appears to be failing tests due to what appears to be memory leak (possibly among other things). Once this gets to the state where all tests are passing, we can take another look.

cbracken avatar Jan 18 '24 18:01 cbracken

@cbracken This is ready for review now.

ardera avatar Feb 21 '24 17:02 ardera

Just to add another data-point, I'm currently working on migrating flutter-rs to the FlutterCompositor API and the EGL surface backing store (added as part of this pull-request) would make the implementation a lot simpler.

So thanks a lot for working on this @ardera and I'm looking forward to using it through the embedder API (once this pull-request lands).

vially avatar Apr 10 '24 18:04 vially

(triage): I spoke to @cbracken and he is going to take another look at this.

goderbauer avatar Jul 10 '24 18:07 goderbauer

I'll see if I can override the CLA bot, but: (a) I'm a Google employee and therefore don't need one. (b) The email address I use for commits is registered in internal systems to tie it to my username. (c) I've had a CLA filed for that email address for years, regardless.

I really suspect this is a bug in the CLA bot, but just for the record it's safe to ignore the CLA bot on this one. As far as I can tell, it only seems to be upset with me when code suggestions I add get committed on other people's PRs. 🤷‍♂️

cbracken avatar Jul 12 '24 17:07 cbracken

I'll see if I can override the CLA bot, but: (a) I'm a Google employee and therefore don't need one. (b) The email address I use for commits is registered in internal systems to tie it to my username. (c) I've had a CLA filed for that email address for years, regardless.

I really suspect this is a bug in the CLA bot, but just for the record it's safe to ignore the CLA bot on this one. As far as I can tell, it only seems to be upset with me when code suggestions I add get committed on other people's PRs. 🤷‍♂️

👍 can also remove the Co-Authored-By if the CLA bot doesn't want to cooperate :)

ardera avatar Jul 12 '24 19:07 ardera

CLA bot is now happy. I have made some changes to the underlying domain -- same one but no longer associated with a Gaia (google) account of that name, so perhaps that's what set it off. Fixing it seems to have involved just re-signing the CLA again.

cbracken avatar Jul 12 '24 19:07 cbracken

I added a SetCurrentResult struct and migrated all the code to that. Is it ready to merge? @chinmaygarde @cbracken

ardera avatar Aug 06 '24 13:08 ardera