cpython
cpython copied to clipboard
gh-142968: Add type checks to token.ISTERMINAL and ISNONTERMINAL
Description
This PR fixes gh-142968 by adding explicit type checks to token.ISTERMINAL and token.ISNONTERMINAL.
Previously, these functions accepted floats (e.g., 0.5) without error, which was inconsistent with PyPy and the expected behavior of token validation. They now raise a TypeError for non-integer inputs, matching the behavior described in the issue.
Changes Made
- Modified
Tools/build/generate_token.pyto inject type checks into the generatedLib/token.py. - Regenerated
Lib/token.pyusing the tool. - Added a new test file
Lib/test/test_token.pywith regression tests for these functions.
Verification
- Validated that
token.ISTERMINAL(0.5)now raisesTypeError. - Ran
python -m test -v test_tokenand confirmed all tests pass.