godot-python
godot-python copied to clipboard
[Godot 4] Better numpy support for image / np.arrays
As @touilleMan himself wrote in one of the issues, there is a considerable amount of developers that are interested in using python in godot for image processing / computer vision and AI or machine learning. Now the communication between numpy standard library and Godots way of handling large byte arrays are not so optimal so there is a room for improvement. This issue is made to collect the ideas, discuss possibilities and usecases for numpy in godot.
My input:
- [ ] More performant communication from opencv / numpy.arrays to godot types
- [ ] functions that encapsulate creating images or streaming audio files generated in python.
I know you commented on my related issue for this (#305) but for the sake of completeness, this is how one can currently go about efficiently converting a Matplotlib canvas (a Numpy Array) to a Godot PoolByteArray:
def canvas_to_godot_image(self, canvas):
image = Image()
width = canvas.get_width_height()[1]
height = canvas.get_width_height()[0]
size = width * height * 3
pool_array = PoolByteArray()
pool_array.resize(size)
with pool_array.raw_access() as ptr:
numpy_array = np.ctypeslib.as_array(ctypes.cast(ptr.get_address(), ctypes.POINTER(ctypes.c_uint8)), (size,))
numpy_array[:] = np.frombuffer(canvas.tostring_rgb(), dtype=np.uint8).reshape(canvas.get_width_height()[::-1] + (3,)).flatten()
image.create_from_data(height, width, False, Image.FORMAT_RGB8, pool_array)
return image
As you wrote, it would be great if godot-python could provide this kind of functionality out of the box.
I think there are two ways in which this could be done:
- Define a PoolByteArray constructor specifically to construct from a Numpy Array, or
- Provide a Numpy-specific util class with a function such as
NumpyUtil.numpy_array_to_pool_byte_array(array)
I think the first one would be more comfortable for users since they don't need to know that Numpy Arrays are a special case with a performant conversion method. However, the second one would be better for the godot-python codebase since it doesn't pollute the core codebase with Numpy-specific functionality.
Alternatively (or in addition), I think both options are realizable with compile options specifying whether Numpy-specific functionality should be compiled in or not - that would be another way of keeping things a bit separate.
@touilleMan what are your thoughts?