devguide icon indicating copy to clipboard operation
devguide copied to clipboard

Document how to add a bytecode specialization

Open Fidget-Spinner opened this issue 11 months ago • 4 comments

How to add a new bytecode specialization is knowledge only a few of us remember right now. We should document this somewhere (devguide or CPython, I don't mind). Hopefully a new contributor can pick this up!

There reference is this PR: https://github.com/python/cpython/pull/116385/files

Assuming you found an instruction that serves as a good specialization candidate. Let's use the example of CONTAINS_OP:

  1. Set the original CONTAINS_OP to a uop by changing it from CONTAINS_OP to _CONTAINS_OP in Python/bytecodes.c. The instruction definition should also change from inst to op.
  2. Add the uop that calls a specializing function. In this case _SPECIALIZE_CONTAINS_OP.
  3. The original CONTAINS_OP is now a new macro consisting of _SPECIALIZE_CONTAINS_OP + _CONTAINS_OP.
  4. Define the cache structure in Include/internal/pycore_code.h. At the very least, a 16-bit counter is needed.
  5. Write the specializing function itself in Python/specialize.c. Refer to any other function in that file for the format.
  6. Remember to update operation stats by calling add_stat_dict in Python/specialize.c.
  7. Add the cache layout in Lib/opcode.py so that Python's dis module will know how to represent it properly.
  8. Bump magic number in Include/internal/pycore_magic_number.h.
  9. Run make regen-all on *nix or build.bat --regen on PC.

Fidget-Spinner avatar Mar 06 '24 17:03 Fidget-Spinner