cpython icon indicating copy to clipboard operation
cpython copied to clipboard

gh-142968: Add type checks to token.ISTERMINAL and ISNONTERMINAL

Open Lakshya-Upadhyaya opened this issue 2 weeks ago • 0 comments

Description

This PR fixes gh-142968 by adding explicit type checks to token.ISTERMINAL and token.ISNONTERMINAL.

Previously, these functions accepted floats (e.g., 0.5) without error, which was inconsistent with PyPy and the expected behavior of token validation. They now raise a TypeError for non-integer inputs, matching the behavior described in the issue.

Changes Made

  • Modified Tools/build/generate_token.py to inject type checks into the generated Lib/token.py.
  • Regenerated Lib/token.py using the tool.
  • Added a new test file Lib/test/test_token.py with regression tests for these functions.

Verification

  • Validated that token.ISTERMINAL(0.5) now raises TypeError.
  • Ran python -m test -v test_token and confirmed all tests pass.

Lakshya-Upadhyaya avatar Jan 03 '26 08:01 Lakshya-Upadhyaya