Replace `ffi.callback()` with `ffi.def_extern()`

[azuwis]

trafficstars

Create a new issue to track the ffi.callback() cause MemoryError on macOS M1.

See comment of @bastibe in https://github.com/bastibe/python-soundfile/issues/325#issuecomment-1048584002

According to https://cffi.readthedocs.io/en/latest/using.html#callbacks, ffi.callback() is old style callback, and has many drawbacks, should be replaced by new style callback def_extern().

I've looked into this, and actually drafted an implementation of the callbacks locally. However, there is a problem with this approach: Currently, we are using the ABI mode of CFFI. The "new style" callbacks you mentioned require that soundfile uses the API mode of CFFI, which means compiling a bespoke version of soundfile for every operating system and version of Python we support.

While I am certainly open to doing this, especially now that we have CI runners to automate the task, I think it is out-of-scope for this pull request. Please raise the issue in a new issue, or preferably, try to draft a pull request for it.

[bastibe]

Thank you for opening this issue! If anyone wants to take a stab at implementing this, I'd be grateful for a pull request.

[f17mkx]

i'm dead

[Liu-Da]

Anyone has some ideas？

[ayghri]

See here https://github.com/bastibe/python-soundfile/pull/338

[hiccup7]

Is there a way to solve the "ffi.callback() cause MemoryError on macOS M1" and allow soundfile to continue to use the ABI mode of CFFI? The CPython and PyPy ecosystems are rapidly evolving. The ABI mode allows a soundfile release to work with new CPython and PyPy versions released later. A large community of users depend on soundfile, and the owner of the soundfile project (@bastibe) may not always have time to release new builds quickly.

[f17mkx]

Yess! Can't remember how though. About 1 more hour of googling fixed it for me. And the fix was quite trivial/ like a non issue. Hope that helps.

[f17mkx]

here: https://github.com/bastibe/python-soundfile/issues/325#issuecomment-1043249345

[f17mkx]

Thank you for opening this issue! If anyone wants to take a stab at implementing this, I'd be grateful for a pull request.

its solved in my comment above. I think you can close this issue (maybe?) I dont know how resolving issues works around here but would feel right to me to do so

[ayghri]

@f17mkx the package from that comment did not work for me. @hiccup7 I'm working with @bastibe to push an update. You can try the wheel here. I tested it on M1 and it works (install via pip)

[luohao123]

@aghriss I got error:

after install your wheel a new error got:

soundfile.py", line 1058, in _open raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name)) soundfile.LibsndfileError: Error opening <_io.BytesIO object at 0x2b90ce360>: Format not recognised.

[luohao123]

@bastibe Can u suppor this? Keep get error on M1

soundfile.py", line 1226, in _init_virtual_io def vio_get_filelen(user_data): MemoryError: Cannot allocate write+execute memory for ffi.callback(). You might be running on a system that prevents this. For more information, see https://cffi.readthedocs.io/en/latest/using.html#callbacks

[ayghri]

@aghriss I got error:

after install your wheel a new error got:

soundfile.py", line 1058, in _open raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name)) soundfile.LibsndfileError: Error opening <_io.BytesIO object at 0x2b90ce360>: Format not recognised.

Sorry I can't help with this. I don't have access to M1 anymore.