aleaxit
Cython question
I have a couple questions about Cython/Sage.
Will the GIL be held when the gmpy2 C API functions are called? (I assume so...)
Should I assume that the GMP/MPIR, MPFR, and MPC libraries will be accessed by multiple OS threads simultaneously? In other words, access to those libraries is not managed by the GIL? (I assume there will be multiple concurrent threads.)
MPFR uses thread-specific (or global if TLS is not supported/enabled) settings to control the exponent ranges. Based on discussions in another issue, when Sage uses MPFR, it sets the the minimum and maximum exponents to the their extreme values. I do the same in gmpy2. But since gmpy2 supports restricted exponent ranges and subnormals, I need to temporarily change the exponent limits and then restore them. (see mpfr_check_range and mpfr_subnormalize)
Since I always restore the exponent range, can I assume the exponent range will remain set to the extremes or should I reset the exponent limits for every call into MPFR? (I'm assuming not and that a Cython program could change the exponents limits so I should reset them. Or I could check the values and raise an exception if the exponent limits are changed but that is only practical if they "shouldn't" be changed.)
The GIL is held in Cython unless explicitely mentioned, see here in Cython doc
Of course, the MPFR could be accessible by multiple threads. As you said it is even intended to be thread safe. I don't understand what you mean with the sentence "access to those libraries is not managed by the GIL". The GIL is just (temporarily) holding execution.
Why would you restore anything in each MPFR call? If MPFR is thread safe, there is no reason for the limits to change. Or I misunderstood something.
My concern is that some extension module (either written using the Cython/gmpy2 interface or a completely distinct module) can run in the same thread as gmpy2 and change the exponent limits.
I will plan to add an optional check of the exponent limits that will raise an exception if they've been changed. If nothing else, it may help with debugging obscure issues.
My concern is that some extension module (either written using the Cython/gmpy2 interface or a completely distinct module) can run in the same thread as gmpy2 and change the exponent limits.
This is indeed possible. But why not consider this a feature as a feature?
I will plan to add an optional check of the exponent limits that will raise an exception if they've been changed. If nothing else, it may help with debugging obscure issues.
What happens if I use MPFR and in the middle of my computations I decide to change the size of exponents. Will there be a crash? If not, I would advocate to just document the fact that the MPFR parameters are bound to a whole thread.
I've given it more thought today and I think the best short-term answer is just documenting the expected use of the exponent settings in MPFR.
The MPFR library uses thread-specific global variables to define the valid exponent range of a result. Both Sage and gmpy2 set the exponent limits to the maximum and minimum values. If a Cython extension needs to change the exponent limits, the previous values should be saved and restored before returning back to Python.
The best long-term solution is to change gmpy2 to save/set/restore the exponent limits for each gmpy2 internal function that calls MPFR/MPC. I'll add this to the list....
I think that documenting is a far better long term solution. To my mind, if something has to be changed it is on the MPFR/MPC side. Did you discuss the matter with them?