pyo3 icon indicating copy to clipboard operation
pyo3 copied to clipboard

Add public methods `compile_string` and `run_code` to the Python struct

Open superserious-dev opened this issue 8 months ago • 4 comments

Add public methods compile_string and run_code to the Python struct and refactor run & eval to use these methods internally.

For code that needs to be evaluated multiple times, compiling the input string and re-running the compiled object is more efficient than recompiling the input string for each run.

Closes #5124.

superserious-dev avatar May 18 '25 05:05 superserious-dev

This is not what @davidhewitt and I intended. I didn't realize you were talking about the Python struct, my apologies for the confusion.

What we'd like is to have these as methods on https://docs.rs/pyo3/latest/pyo3/types/struct.PyCode.html, with compile as a constructor (so, that returns Bound<'_, PyCode>), and then a run method to run the code object. You should be able to downcast the result of the Py_Compile* family into a PyCode object.

mejrs avatar May 26 '25 21:05 mejrs

@mejrs Thank you for the clarification. A few questions:

  • Will PyCode::compile and PyCode::run need take in a python struct, eg py: Python<'py>? It looks Bound will require it.
  • Should PyCode::compile return Bound<'_, PyCode> or PyResult<Bound<'_, PyCode>>? It looks like downcasting is fallible, however downcast_into_unchecked could be used to bypass the fallibility.
  • Does this logic from the Python struct's run need to be replicated in PyCode::run? https://github.com/PyO3/pyo3/blob/8373f451fdf9caf6b6ed3cb57b576aac067bad07/src/marker.rs#L612-L651

superserious-dev avatar Jun 01 '25 23:06 superserious-dev

Will PyCode::compile and PyCode::run need take in a python struct, eg py: Python<'py>? It looks Bound will require it

Yes

Should PyCode::compile return Bound<', PyCode> or PyResult<Bound<', PyCode>>? It looks like downcasting is fallible, however downcast_into_unchecked could be used to bypass the fallibility.

Yeah it (and run) should return a result. Not (just) for the downcasting but because the compile function itself can fail if users pass in malformed code or code that raises exceptions.

Does this logic from the Python struct's run need to be replicated in PyCode::run?

We'd prefer not to have duplication if possible. One should call the other.

mejrs avatar Jun 02 '25 12:06 mejrs

@mejrs I did a another pass on this to address the feedback. I created run_code_object because no existing method on the Python struct will accept a code object. Please let me know if this is closer to what you are looking for.

superserious-dev avatar Jun 08 '25 23:06 superserious-dev

I looked into this because PyCode is currently not unconditionally available, which does make implementing this not quite straight forward. While playing around trying to solve that, I ended up with a solution. I did not really intent to fully take this over, I'm sorry about that, but given that I ended up with something working I thought I might as well put it up for discussion in #5217

Icxolu avatar Jun 28 '25 20:06 Icxolu

Hello @Icxolu. No problem. It sounds like you have a good understanding of this part of the codebase; i'll close this PR and let you carry forward with solution.

superserious-dev avatar Jun 30 '25 19:06 superserious-dev