beartype
[Feature Request] Allow emitting warnings rather than throwing errors for guadual adoption
We are looking to adopt beartype for a library project. Since there are many existing users, turning on beartype at once may cause existing code that depend on the library to break (because of inexact type annotations and unexpected usage). I am hoping to find a way to make beartype produce warnings than erroring to allow us test the type annotations and avoid breaking user code.
Is it already possible to make beartype produce warnings when the types do not match; are there good ways of doing this? Thanks!
Excellent idea, QA fox. And allow me to humbly apologize for my slooooooooow reply. I'm on the cusp of releasing beartype 0.11.0 and... the commit churn got unexpectedly hectic there.
But this is a fantastic feature request that we should absolutely implement as soon as possible. Sadly, this probably won't make it in time for beartype 0.11.0. In the meanwhile, if your wonderful team wouldn't mind implementing this on your end, everything you want to happen should be readily achievable by this mostly trivial decorator that I am calling bearcubtype: ...heh, heh, heh
from beartype import beartype
from beartype.roar import BeartypeCallHintViolation
from beartype.typing import TypeVar
from warnings import warn
T = TypeVar('T')
def bearcubtype(beartypeable: T) -> T:
beartyped = beartype(beartypeable)
def _coerce_beartype_exceptions_to_warnings(*args, **kwargs) -> object:
try:
return beartyped(*args, **kwargs)
except BeartypeCallHintViolation as violation:
warn(violation)
return _coerce_beartype_exceptions_to_warnings
Then just do this:
>>> @bearcubtype
... def muh_func(muh_arg: str) -> int:
... return len(muh_arg)
>>> muh_func(42)
UserWarning: @beartyped __main__.muh_func() parameter
muh_arg=42 violates type hint <class 'str'>, as int 42 not instance of str.
warn(violation)
Viola! As you ask, so to do you receive. Instant non-fatal warnings where once there were only fatal exceptions. Does that slake your momentary thirst for gradual typing and adoption? Oh – and thanks for all the warm enthusiasm! May your team find endless success with foxy runtime type-checking. :fox_face:
Love this and thank you for your response! In the function you shared, I noticed that a warning is emitted (great!) but the function was not run.
We were looking to handle a case like this, where a function can handle the input but is annotated incorrectly due to our own negligence:
# Not so pythonic function just as an example
@bearcubtype
def to_tuple(l: List[str]) -> Tuple[str]:
return tuple(elem for elem in l)
# Works
to_tuple(["a", "b"])
# Should work but fails
to_tuple([42, 1])
to_tuple(("a", "b"))
We hope that bear can warn us when we call functions with the wrong types of the arguments, but still allow the function to run and return if Python doesn't complain. We can then act on the warnings to make sure the types are added correctly.
I can try to prototype this in a PR. My current plan is to see if I can condition on where the roars happen and warn + return instead. Do you have any suggestions?
Oh – you're totally right. Let's make bearcubtype even more resilient against type-checking failures:
from beartype import beartype
from beartype.roar import BeartypeCallHintViolation
from beartype.typing import TypeVar
from warnings import warn
T = TypeVar('T')
def bearcubtype(beartypeable: T) -> T:
beartyped = beartype(beartypeable)
def _coerce_beartype_exceptions_to_warnings(*args, **kwargs) -> object:
try:
return beartyped(*args, **kwargs)
except BeartypeCallHintViolation as violation:
warn(violation)
return beartypeable(*args, **kwargs)
return _coerce_beartype_exceptions_to_warnings
Admittedly untested... but probably works. The decorated callable is now guaranteed to be called regardless of whether type-checking emits a non-fatal warning or not. Is that a reasonable compromise?
What We Gonna Do Now Is...
This is a surprisingly fun issue! Although trivial, the above approach inefficiently adds an additional stack frame to the call stack. That's a bit non-ideal. Our whole modus operandi here at Beartype, Inc. is ludicrous speed, right?
I suspect we'll end up implementing this in @beartype proper as a new beartype.BeartypeConf boolean parameter. When enabled, that parameter will directly instruct our dynamic code generation machinery to emit a type-checking wrapper that emits warnings rather than raises exceptions: e.g.,
# Something like this is what I'm sayin'.
from beartype import beartype, BeartypeConf
@beartype(conf=BeartypeConf(is_violation_warn=True))
def foxy_func(foxy_arg: int) -> str:
return str(foxy_arg)
Wonderful end users such as yourself can then either set that boolean parameter directly (as above) or define a wrapper with functools.partial to automate that for you (as below):
# Same thing as above, but with a more compact and reusable decorator.
from beartype import beartype, BeartypeConf
from functools import partial
bearcubtype = partial(beartype, conf=BeartypeConf(is_violation_warn=True))
@bearcubtype
def foxy_func(foxy_arg: int) -> str:
return str(foxy_arg)
Implementing support for is_violation_warn means more effort on our part, which could be a blocker. But that avoids the additional stack frame of our initial approach, which makes everything worthwhile. It's "Go fast or go home!" here at @beartype. :wink:
Thanks yet again for all the exuberant enthusiasm, @justinchuby. Foxes rock.
Wonderful! That's probably good enough for our use now. To make sure it works in all cases, does beartype check for return types as well? If it does, I wonder how we should be dealing with functions that produces side effects (most of ours).
edit:
with a little bit of testing, I realized:
k = 0
@bearcubtype
def spell(x: str) -> int:
global k
k += 1
return str(x)
print(spell("x"))
print(k)
x
2
As a compromise, maybe I can create a wrapper that checks for only the input types with is_bearable?
...heh. Yet again, you're absolutely right. I dub thee @alwaysrightjustinchuby! :magic_wand:
As a temporary workaround, the beartype.abby subpackage is indeed your best bet. Always bet on the Bear Abby. Given that existing functionality, consider defining your own non-fatal warn_if_unbearable() runtime type-checker ala:
from beartype.abby import die_if_unbearable
from beartype.roar import BeartypeCallHintViolation
from warnings import warn
def warn_if_unbearable(obj: object, hint: object) -> None:
try:
die_if_unbearable(obj, hint)
except BeartypeCallHintViolation as violation:
warn(violation)
k = 0
@bearcubtype
def spell(x: str) -> object: # <-- return obfuscated to avoid double type-checks
global k
k += 1
return_value = str(x)
warn_if_unbearable(return_value, int)
return return_value
print(spell("x"))
print(k)
...which then exhibits the expected behaviour:
/home/leycec/tmp/mopy.py:30: UserWarning: Object 'x' violates type hint <class 'int'>, as str 'x' not instance of int.
warn(violation)
x
1
So... That's a Bit Awkward
Yeah. Definitely awkward, but probably the closest approximation to your vision statement that @beartype can currently get. Without full-blown support for the beartype.BeartypeConf(is_violation_warn=True) parameter proposed above, it's hard to envision a cleaner approach.
Once implemented, that parameter will "...directly instruct our dynamic code generation machinery to emit a type-checking wrapper that emits warnings rather than raises exceptions." That, in turn, would avoid all of the unpleasant edge cases you stumbled into above. :face_exhaling:
OMG. I just stupidly automated everything away by combining the unwholesome powers of @bearcubtype + warn_if_unbearable(). Behold! It sickens me even as it solves all your problems:
from beartype import beartype
from beartype.abby import die_if_unbearable
from beartype.roar import BeartypeCallHintViolation
from beartype.typing import TypeVar
from warnings import warn
T = TypeVar('T')
def bearcubtype(beartypeable: T) -> T:
hint_return = beartypeable.__annotations__.get('return')
if hint_return is not None:
del beartypeable.__annotations__['return']
beartyped = beartype(beartypeable)
def _coerce_beartype_exceptions_to_warnings(*args, **kwargs) -> object:
try:
obj_return = beartyped(*args, **kwargs)
if hint_return is not None:
try:
die_if_unbearable(obj_return, hint_return)
except BeartypeCallHintViolation as violation:
warn(violation)
return obj_return
except BeartypeCallHintViolation as violation:
warn(violation)
return beartypeable(*args, **kwargs)
return _coerce_beartype_exceptions_to_warnings
k = 0
@bearcubtype
def spell(x: str) -> int:
global k
k += 1
return str(x)
print(spell("x"))
print(k)
exit(0)
..which still exhibits the expected behaviour:
/home/leycec/tmp/mopy.py:26: UserWarning: Object 'x' violates type hint <class 'int'>, as str 'x' not instance of int.
warn(violation)
x
1
Let's pretend this never happened. Everything above should work, but @bearcubtype is rapidly trending into the Outer Limits of Runtime Type-checking – which explains why you probably don't want to try packaging that into a well-tested PR for us. Personally, I wouldn't advise that. At this point, it's best to just run and take the money. :smiling_face_with_tear:
This is fantastic, thank you so much for this! 😄 I played with the snippet and settled on what's below, where I decided the return type is not so important at the moment; in the warnings I print out the stacktrace as well.
I am happy to contribute back if you see there is a good way for me to do it.
import functools
import warnings
import traceback
from beartype import beartype, roar as _roar
class CallHintViolationWarning(UserWarning):
pass
def bearcubtype(beartypeable):
if "return" in beartypeable.__annotations__:
del beartypeable.__annotations__["return"]
beartyped = beartype(beartypeable)
# Does beartype make a copy of the __annotations__? Can I add return back to beartypeable here?
@functools.wraps(beartyped)
def _coerce_beartype_exceptions_to_warnings(*args, **kwargs):
try:
return beartyped(*args, **kwargs)
except _roar.BeartypeCallHintViolation:
# Fall back to the original function if the beartype hint is violated.
warnings.warn(
traceback.format_exc(), category=CallHintViolationWarning, stacklevel=2
)
return beartypeable(*args, **kwargs)
return _coerce_beartype_exceptions_to_warnings
k = 0
@bearcubtype
def spell(x: str) -> int:
global k
k += 1
return str(x)
print(spell("x"))
print(spell(42))
print(k)
x
<ipython-input-11-9d61806e0a37>:10: CallHintViolationWarning: Traceback (most recent call last):
File "<ipython-input-10-586af291e575>", line 20, in _coerce_beartype_exceptions_to_warnings
return beartyped(*args, **kwargs)
File "<@beartype(__main__.spell) at 0x102923910>", line 21, in spell
File "venv/lib/python3.10/site-packages/beartype/_decor/_error/errormain.py", line 301, in raise_pep_call_exception
raise exception_cls( # type: ignore[misc]
beartype.roar.BeartypeCallHintParamViolation: @beartyped spell() parameter x=42 violates type hint <class 'str'>, as 42 not instance of str.
print(spell(42))
42
2
More impressive questions demand equally impressive answers:
# Does beartype make a copy of the __annotations__?
Nope, but great thought. For efficiency, beartype directly inspects – and, under certain common edge cases (e.g., PEP 585-compliant type hints), can even directly modify – the __annotations__ dictionary of the decorated callable. That said...
# Can I add return back to beartypeable here?
Yeah! This is the thing to do. For safety, you'll want to wrap everything in a try: ...; finally: ... block to ensure that the return type hint is added back even when @beartype raises an exception (e.g., due to a type hint being invalid or unsupported). So, something resembling:
# Also realized you need a "SENTINEL" placeholder. Why?
# Because "None" is a common valid type hint. Edge cases! *sigh*
_SENTINEL = object()
def bearcubtype(beartypeable):
hint_return = beartypeable.__annotations__.get('return', _SENTINEL)
if hint_return is not _SENTINEL:
del beartypeable.__annotations__["return"]
try:
beartyped = beartype(beartypeable)
finally:
if hint_return is not _SENTINEL:
beartypeable.__annotations__["return"] = hint_return
@functools.wraps(beartyped)
def _coerce_beartype_exceptions_to_warnings(*args, **kwargs):
try:
return beartyped(*args, **kwargs)
except _roar.BeartypeCallHintViolation:
# Fall back to the original function if the beartype hint is violated.
warnings.warn(
traceback.format_exc(), category=CallHintViolationWarning, stacklevel=2
)
return beartypeable(*args, **kwargs)
return _coerce_beartype_exceptions_to_warnings
So exciting to see you pushing @beartype hard into @pytorch, too. I will finally choose to believe in a higher computational power when your PR goes through. :smile:
Looks like adding hint_return back causes k to become 3 in
k = 0
@bearcubtype
def spell(x: str) -> int:
global k
k += 1
return str(x)
print(spell("x"))
print(spell(42))
print(k)
(should be 2)
Huh. Would you look at that! Sadly, I can't replicate the badness you're seeing on my end. This...
from beartype import beartype
from beartype.abby import die_if_unbearable
from beartype.roar import BeartypeCallHintViolation
from beartype.typing import TypeVar
from warnings import warn
T = TypeVar('T')
import functools, traceback, warnings
from beartype import beartype, roar as _roar
class CallHintViolationWarning(UserWarning):
pass
_SENTINEL = object()
def bearcubtype(beartypeable):
hint_return = beartypeable.__annotations__.get('return', _SENTINEL)
if hint_return is not _SENTINEL:
del beartypeable.__annotations__["return"]
try:
beartyped = beartype(beartypeable)
finally:
if hint_return is not _SENTINEL:
beartypeable.__annotations__["return"] = hint_return
@functools.wraps(beartyped)
def _coerce_beartype_exceptions_to_warnings(*args, **kwargs):
try:
return beartyped(*args, **kwargs)
except BeartypeCallHintViolation:
# Fall back to the original function if the beartype hint is violated.
warnings.warn(
traceback.format_exc(), category=CallHintViolationWarning, stacklevel=2
)
return beartypeable(*args, **kwargs)
return _coerce_beartype_exceptions_to_warnings
k = 0
@bearcubtype
def spell(x: str) -> int:
global k
k += 1
return str(x)
print(spell("x"))
print(spell(42))
print(k)
exit(0)
...thankfully still gives the expected output for me, I think?
x
/home/leycec/tmp/mopy.py:52: CallHintViolationWarning: Traceback (most recent call last):
File "/home/leycec/tmp/mopy.py", line 33, in _coerce_beartype_exceptions_to_warnings
return beartyped(*args, **kwargs)
File "<@beartype(__main__.spell) at 0x7fc182246b90>", line 21, in spell
beartype.roar.BeartypeCallHintParamViolation: @beartyped __main__.spell() parameter x=42 violates type hint <class 'str'>, as int 42 not instance of str.
print(spell(42))
42
2
Is that not what you're seeing? Or... am I missing something? This might be a global state issue on your side. If you're still in Jupyter Notebook or IPython, your kernel might benefit from being restarted. Maybe? I'd give that a go first, anyway.
That escalated quickly. Thanks again for bearing with me over so many comments. Pretty sure we've finally got @bearcubtybe nailed down real tight, now. :hammer:
No worries, whatsoever. This is the darkest code voodoo. I wouldn't be surprised if @bearcubtype spontaneously erupts in putrid green flames when prodded gently with an edge case like multithreading. For now, let's pretend everything works so that I can sleep tonight. :sleepy:
Hi all,
was there any additional progress on this feature request? While the solution with the custom decorator would work for me, I looking forward to use the upcoming claw feature and wonder how this will play together?
Excellent question! Sadly, you will hate the answer – which is that the first incarnation of the beartype.claw API in our upcoming @beartype 0.15.0 release will forcefully raise exceptions on type-checking violations. Gradual typing is probably right out... for the moment.
I share your pain and apologize profusely for it. Ordinarily, you'd be able to intervene by leveraging your own hand-rolled decorator like @bearcubtype above. But beartype.claw is beyond ordinary; there's really no means of gracefully intervening there, short of awkward monkey patches of private @beartype functionality. Nobody wants to (or should) go there.
Technically, beartype.claw does permit some level of fine-grained control over import hook behaviour; for example, @beartype 0.15.0 introduces a new BeartypeConf.is_claw_pep526 parameter enabling users to conditionally disable runtime type-checking of PEP 526-compliant annotated variable assignments (e.g., muh_var: int = 'Pretty sure this isn't an int.'). But beartype.claw currently lacks similar support for controlling gradual typing. I know, right? That's awful.
The Shape of the Future To Come: It's a Giant Bear
I failed to realize that somebody ...perhaps many somebodies wanted gradual typing in their import hooks. But fear not! The hypothetical BeartypeConf.is_violation_warn parameter proposed waaaay above could very well land in @beartype 0.16.0. If so, beartype.claw would hopefully support that out-of-the-box as well.
I should probably admit that my immediate priority for @beartype 0.16.0 is deep type-checking of set and dict containers, which everyone wants and is ready to crucify a burning effigy of my GitHub avatar over. I agree with them (minus that crucification part). It's kinda embarrassing that we haven't done that yet. My forehead is beaded with sweat.
Let us see what the dismal future holds for us all. :crystal_ball:
Finally nailed it. Thanks so much for your extreme patience, @justinchuby and @kasium. Due to a recent PR by Quebec Big Brain @felix-hilden + extensive refactoring on the @beartype backend in c788591e5fceda, our upcoming @beartype 0.17.0 release now fully supports gradual typing.
Specifically, the beartype.BeartypeConf dataclass now supports two new options for configuring type-checking violations:
-
violation_param_type, the type of exception to be raised on type-checking an invalid parameter. -
violation_return_type, the type of exception to be raised on type-checking an invalid return.
Crucially, warnings are exceptions in Python. Literally. The builtin Warning class subclasses the builtin Exception class. Moreover, @beartype explicitly detects when you are attempting to emit a non-fatal warning by passing a Warning subclass for one of the above two options.
In short, this means that the following now does what everybody wants:
# In your "your_package.__init__" submodule:
from beartype import BeartypeConf
from beartype.claw import beartype_this_package
beartype_this_package(conf=BeartypeConf(
violation_param_type=UserWarning,
violation_return_type=UserWarning,
))
Of course, you can pass an app-specific Warning subclass rather than UserWarning. And... you should probably do that. :laughing:
@beartype: "Now you're playing with QA Bear Power."
Resolved by b5ed824. Gradually adopt a QA bear cub today, the @beartype way:
# In your "your_package.__init__" submodule:
from beartype import BeartypeConf
from beartype.claw import beartype_this_package
beartype_this_package(conf=BeartypeConf(
violation_param_type=UserWarning,
violation_return_type=UserWarning,
))
Thanks yet again, @justinchuby and @kasium. This flexed bear bicep is for you. :muscle: :bear:
Thanks a lot! I looking forward to the next release which contains the feature