Pablo

Also, make RunnerContextManager fully immutable (will also help to persist it with ray.put) to avoid the cost of passing it around ever and over.

I think it's fine if @flyingImer picks this up ? : )

theres's very silly superficial stuff I wrote here: https://docs.google.com/document/d/1DcuKhCPnZezIvu9vFMsM4BRdBv0kgAWewOJqRbS42GI/edit# Specifically, I would say `read_datasource` may be a good point to look at - it spins up several Ray Tasks that...

Beam has a few utilities to convert to-from Beam and Arrow schemas (see [here](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/typehints/arrow_type_compatibility.py)). A first step would be to write an `ArrowRecordBatchCoder`, which can be constructed with a Beam...

I'll take on this issue as well.

@valiantljk I will get some pointers to look at this : )

hmmm thanks for finding that. I'll try to debug this week...

This should clean up all Ray resouces after every test: https://github.com/ray-project/ray_beam_runner/pull/40/ Note there's the IMPULSE element that we were creating in a 'global' context.

Hello team. I'm interested in picking up this item. I'll try and do the relevant research and propose a minimum design in the next few days (realistically weeks:))

Hi! Thanks for filing the issue. Would you like to submit a pull request with the change so I can pull it in?