Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vendor chill #5602

Open
wants to merge 16 commits into
base: v0.15.x
Choose a base branch
from
Open

Vendor chill #5602

wants to merge 16 commits into from

Conversation

kellen
Copy link
Contributor

@kellen kellen commented Feb 10, 2025

I looked into whether importing some subset of chill was reasonable but most of the various chill deps are kryo serializers or registrars, so appear to be required for our usecase.

@kellen
Copy link
Contributor Author

kellen commented Feb 27, 2025

Punting to 0.16

@kellen kellen added this to the 0.16.0 milestone Feb 27, 2025
@jto
Copy link
Contributor

jto commented Mar 12, 2025

Wouldn't it be easier to just get rid of Chill ?

A lot of usage in Scio code are references to types which are just aliases:

type Kryo = com.esotericsoftware.kryo.Kryo
type KSerializer[T] = com.esotericsoftware.kryo.Serializer[T]
type Input = com.esotericsoftware.kryo.io.Input
type Output = com.esotericsoftware.kryo.io.Output

Then there's ClosureCleaner but it's has no dependency on the rest of chill and could be copied in Scio or we could use another implementation.

We also rely on Externalizer but again that's 200 loc and potentially we could re-implement it.

The biggest issue perhaps would be the loss of Kryo coders for Scala types but since we barely use Kryo anymore maybe it's ok to not have them ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants