ironpython3 icon indicating copy to clipboard operation
ironpython3 copied to clipboard

ctypes .from_buffer only works for array.array

Open orent opened this issue 2 years ago • 10 comments

The .from_buffer[_copy] methods of ctypes types only accept an array.array argument. Other implementations such as cpython, pypy also accept bytes, bytearray, mmap, memoryview etc.

The struct and re module accept a wider variety of buffer-like objects. Strangely, re accepts mmap objects, but stuct does not.

orent avatar Apr 17 '22 15:04 orent

Correction: from_buffer_copy() does accept bytes, but not bytearray, memoryview, etc.

orent avatar Apr 17 '22 15:04 orent

Thanks for the submission. This seems to be related to #21, which is only partially finished. What is still needed is a review of all modules and converting their API to use IBufferProtocol as needed. struct and ctypes are the two prominent ones. Especially getting ctypes up to speed will be exciting because it is a prerequisite for supporting numpy.

BCSharp avatar Apr 17 '22 18:04 BCSharp

from_buffer is probably blocked by https://github.com/IronLanguages/ironpython3/issues/1297. from_buffer_copy should be simple to fix with IBufferProtocol (thanks to @BCSharp).

slozier avatar Apr 17 '22 19:04 slozier

Yes, from_buffer_copy will be simple. I can pick it up since the int/long unification stuff is drying up. For the from_buffer, indeed #1297 needs to be done first.

BCSharp avatar Apr 17 '22 19:04 BCSharp

@BCSharp I guess I should have waited a bit! https://github.com/IronLanguages/ironpython3/pull/1405 could use a review.

slozier avatar Apr 17 '22 19:04 slozier

No problem! I wouldn't be able to get at it till tonight at best. Will review your PR instead, hopefully this time pushing all the right buttons... 😄

BCSharp avatar Apr 17 '22 20:04 BCSharp

Thanks for the fast response. Two more minor inconveniences that caused my code not to work out-of-the-box on IronPython are:

  1. struct.unpack_from not accepting an mmap object
  2. ctypes.Structure._pack_ attribute accepts only int, not bool

Should I open issues for these?

what is the easiest way to get a build of something newer than 3.4alpha?

orent avatar Apr 18 '22 12:04 orent

Should I open issues for these?

I think struct.unpack_from will take a bit of effort to implement properly and might be worth its own issue. I can take a look at ctypes.Structure._pack_ later today, I think it should be easy.

what is the easiest way to get a build of something newer than 3.4alpha?

You should be able to get the latest build from the main page. If you search for "builds" there are badges for the Azure and Github CI. They should both publish build artifacts that you can download.

slozier avatar Apr 18 '22 13:04 slozier

After giving it some thought I am now becoming more convinced that #1297 is not the way to go here. In the coming days I will do some experiments for an alternative solution (like pinning supported by IPythonBuffer). If it goes somewhere I will submit a draft PR to elicit comments.

BCSharp avatar Apr 18 '22 17:04 BCSharp

It's confusing, because all the newly built packages are still named "3.4alpha", just like the April 2021 release.

On Mon, 18 Apr 2022 at 16:09, slozier @.***> wrote:

Should I open issues for these?

I think struct.unpack_from will take a bit of effort to implement properly and might be worth its own issue. I can take a look at ctypes.Structure.pack later today, I think it should be easy.

what is the easiest way to get a build of something newer than 3.4alpha?

You should be able to get the latest build from the main page. If you search for "builds" there are badges for the Azure and Github CI. They should both publish build artifacts that you can download.

— Reply to this email directly, view it on GitHub https://github.com/IronLanguages/ironpython3/issues/1404#issuecomment-1101395628, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF2ID3NNWVWCUCEBGHDZKTVFVNJJANCNFSM5TUADXRA . You are receiving this because you authored the thread.Message ID: @.***>

orent avatar Apr 19 '22 15:04 orent