kafka-python icon indicating copy to clipboard operation
kafka-python copied to clipboard

Failures in struct packing should fail and not hang forever

Open jeffwidman opened this issue 8 years ago • 2 comments

If we try to build a struct using incorrect types, then _pack() is correctly raising an error. However, the foreground thread just hangs when this happens.

This doesn't make sense to me--it's a client side error, so we should be able to fail loudly to the user. The error may be slightly cryptic because all the struct knows is that something is incorrect, it has no idea of the broader context. But at least it'd give the user something to go on...

This behavior may also apply to _unpack().

To reproduce, in _pack(), manually insert a conditional that flips a type:

def _pack(f, value):
    try:
        if isinstance(value, int):
            value = str(value)
        return pack(f, value)
    except error:
        raise ValueError(error)

See also #1318

jeffwidman avatar Dec 08 '17 00:12 jeffwidman

@jeffwidman Just to confirm, we talk about the producer here, yes?

tvoinarovskyi avatar Dec 08 '17 07:12 tvoinarovskyi

I only tested on the producer. As best I can tell, this hanging is a result of the interaction between threads. So the current version of the consumer, being single-threaded, will not have this problem. However, the addition of the background thread in #1266 may introduce this for the consumer as well... I haven't tested that.

jeffwidman avatar Dec 08 '17 09:12 jeffwidman