godot icon indicating copy to clipboard operation
godot copied to clipboard

[3.x] Update texture image data from a byte buffer

Open kisg opened this issue 1 year ago • 5 comments

When a texture needs to be updated frequently (e.g. every frame) converting the texture data first to an image causes too many delays. The new APIs allow the applications to update the image data of a texture directly from a byte array. An offset parameter makes it possible to only use a part of the byte array for the update.

Companion proposal: https://github.com/godotengine/godot-proposals/issues/5342

kisg avatar Sep 05 '22 11:09 kisg

Use case: I know people using Python bindings have been looking for ways to interop opencv images, which IIRC are just numpy arrays under the hood, so bytes. This addition should make them very happy

Zireael07 avatar Sep 05 '22 12:09 Zireael07

There should be no delay, the pool arrays are copy-on-write if I am not mistaken, so this should not affect performance.

reduz avatar Sep 16 '22 20:09 reduz

There should be no delay, the pool arrays are copy-on-write if I am not mistaken, so this should not affect performance.

In this PR we are using pool arrays to store the byte buffers. The delay comes from the pool array -> image -> ImageTexture -> VisualServer::texture_set_data() sequence, where the texture_set_data() function examines the image formats every single time. This PR adds a shortcut, where the pool array data can be directly uploaded to the GL driver without going through all the Image conversion / examination path.

kisg avatar Sep 18 '22 18:09 kisg

Use case: I know people using Python bindings have been looking for ways to interop opencv images, which IIRC are just numpy arrays under the hood, so bytes. This addition should make them very happy

In OpenCV you can actually take an external buffer (e.g. a pool array) and wrap it as a cv::Mat and use it either as a source or as a target buffer. We used this in our project.

kisg avatar Sep 18 '22 19:09 kisg

direct read from a numpy ndarray onto a sprite texture would be nice too

TsiamDev avatar Mar 25 '23 19:03 TsiamDev