Vendor chill

Open kellen opened this issue 10 months ago • 2 comments

I looked into whether importing some subset of chill was reasonable but most of the various chill deps are kryo serializers or registrars, so appear to be required for our usecase.

Feb 10 '25 21:02 kellen

Punting to 0.16

Feb 27 '25 20:02 kellen

Wouldn't it be easier to just get rid of Chill ?

A lot of usage in Scio code are references to types which are just aliases:

type Kryo = com.esotericsoftware.kryo.Kryo
type KSerializer[T] = com.esotericsoftware.kryo.Serializer[T]
type Input = com.esotericsoftware.kryo.io.Input
type Output = com.esotericsoftware.kryo.io.Output

Then there's ClosureCleaner but it's has no dependency on the rest of chill and could be part of Scio or we could use another implementation.

We also rely on Externalizer but again that's 200 loc and potentially we could re-implement it.

The biggest issue perhaps would be the loss of Kryo coders for Scala types but since we barely use Kryo anymore maybe it's ok to not have them ?

Mar 12 '25 10:03 jto