softbuffer icon indicating copy to clipboard operation
softbuffer copied to clipboard

WIP: Attempt to use `IOSurface` on macOS

Open ids1024 opened this issue 2 years ago • 8 comments

It looks like BytesPerRow (the stride) has some kind of alignment requirement (testing on M1), that doesn't let us just use the width. So for this to work, softbuffer needs to expose the stride, and the user needs to deal with that...

This doesn't seem to be showing the right thing even when width matches stride. Not sure why.

ids1024 avatar Apr 10 '23 22:04 ids1024

This doesn't seem to be showing the right thing even when width matches stride. Not sure why.

Ah, so this part is due to the alpha channel. https://developer.apple.com/documentation/corevideo/cvpixelformatdescription/1563591-pixel_format_identifiers doesn't list an BGRX format or similar, so it looks like we need to set the alpha of each pixel to 255 for it to display properly.

ids1024 avatar Apr 10 '23 22:04 ids1024

We'll probably need to expose a stride to efficiently implement https://github.com/rust-windowing/softbuffer/issues/44 using https://developer.android.com/ndk/reference/struct/a-native-window-buffer#struct_a_native_window___buffer

So I think we'll ultimately want an API change like this.

ids1024 avatar Apr 11 '23 20:04 ids1024

We'll probably need to expose a stride to efficiently implement #44 using https://developer.android.com/ndk/reference/struct/a-native-window-buffer#struct_a_native_window___buffer

So I think we'll ultimately want an API change like this.

I would highly recommend looking into imgref for this.

LoganDark avatar Aug 25 '23 15:08 LoganDark

Is imgref currently used in any crates that would likely be used with Softbuffer for rendering?

We need the Buffer type to provide a present method, and on some backends to release certain resources on drop. So it's natural enough to provide a stride() method on that. But we could add a method returning ImgRefMut if that will be useful to users of softbuffer.

ids1024 avatar Aug 29 '23 16:08 ids1024

Should we move this discussion over to #98 which is now open to discuss the API considering requirements of various platforms, not just limiting it to macOS here?

FWIW such a crate could be provided behind a [feature] gate if desired. I've open-coded a similar thing when adding the necessary bindings to the NDK to facilitate support in softbuffer (#130): https://docs.rs/ndk/0.8.0-beta.0/ndk/native_window/struct.NativeWindowBufferLockGuard.html#method.lines

MarijnS95 avatar Aug 30 '23 07:08 MarijnS95

Is imgref currently used in any crates that would likely be used with Softbuffer for rendering?

We need the Buffer type to provide a present method, and on some backends to release certain resources on drop. So it's natural enough to provide a stride() method on that. But we could add a method returning ImgRefMut if that will be useful to users of softbuffer.

The Buffer should deref to ImgRefMut instead of &[u32]. That's what I mean. Then the ImgRefMut itself would know its own stride.

Bonus if you do ImgRefMut<ARGB> instead of ImgRefMut<u32> but either way is fine.

LoganDark avatar Aug 30 '23 15:08 LoganDark

Bonus if you do ImgRefMut<ARGB> instead of ImgRefMut but either way is fine.

Yeah, that part gets back to https://github.com/rust-windowing/softbuffer/issues/98. Unfortunately it seems we need to use RGBA on some platforms for no-copy presentation....

So I think we've settled on this solution for macOS, it's just that it requires breaking API changes, and we may want to do that along with https://github.com/rust-windowing/softbuffer/issues/98.

ids1024 avatar Aug 30 '23 15:08 ids1024

Unfortunately it seems we need to use RGBA on some platforms for no-copy presentation

Well right now you say u32. Using the dedicated structs from rgb crate would be better and it's even what I use internally before converting to u32. So in a way you just proven my point

LoganDark avatar Aug 30 '23 15:08 LoganDark