Dan King
Dan King
Daniel Marten will collaborate with Chris Vittal on this project. Daniel Marten will reach out to find a time to meet with Chris. Goal is to pair program to implement...
Very exciting!
Apologies for delay; I will look this afternoon.
This all seems fine to me now. I'm skeptical the new types will catch non-trivial bugs, but they're also not syntactically obtrusive. I'm strongly opposed to unnecessary writing and reading...
[batch] Submitting a Batch using the hailtop.batch library may raise "Cannot enter into task" errors
Possibly related: https://github.com/erdewit/nest_asyncio/issues/22#issuecomment-1300570745 If Ben W is using asyncio in *his* code, it seems likely we'll end up with unpatched tasks. If this is the problem, we should expose an...
[batch] Submitting a Batch using the hailtop.batch library may raise "Cannot enter into task" errors
Simple replication: ``` In [6]: import hailtop.batch as hb ...: b = hb.Batch(backend=hb.ServiceBackend()) ...: for _ in range(300): ...: j = b.new_job() ...: j.command(f'echo {"a" * 11 * 1024}') ...:...
[batch] Submitting a Batch using the hailtop.batch library may raise "Cannot enter into task" errors
Hmm. It doesn't replicate with that alone. It only replicates after I ran some other code.
[batch] Submitting a Batch using the hailtop.batch library may raise "Cannot enter into task" errors
This seems to do it: ``` In [1]: import hailtop.batch as hb ...: b = hb.Batch(backend=hb.ServiceBackend()) ...: for _ in range(32): ...: j = b.new_job() ...: j.command(f'cat >/dev/null {" ".join(b.read_input("gs://danking/foo.vcf")...
[batch] Submitting a Batch using the hailtop.batch library may raise "Cannot enter into task" errors
Hmm. Not reliably.
[batch] Submitting a Batch using the hailtop.batch library may raise "Cannot enter into task" errors
Sayonara! I'm pretty sure this is yet another consequence of ipython's event loop shenanigans: - https://github.com/hail-is/hail/issues/14099 - https://github.com/hail-is/hail/issues/14166 - https://github.com/ipython/ipython/issues/14297