Chris Lu

I mean the user can adjust the exported setting file.

If you plan to do the window functions, what are you thinkings for the underlying data structures? need your email to add you to slack.

The API changes have some good ideas, but need more thoughts on * how to handle late events * how to run blocking functions, such as sort, on the windowed...

Thanks for the PR! How about adding a new type of Mapper, e.g., MapperWithArgs, to avoid incompatible changes?

> And what would be the signature of `Map`? Would we add a `MapWithArgs` also? Yes, that seems cleaner.

yes. a big internal change. Actually I am also thinking to provide mapper and reducer objects that can have internal states. This is something related, and can be re-engineered together.

Yes, your proposal looks fine, even though it adds another set of functions.

@xitongsys could you please help (again) on updating the parquet APIs? It will be nice not to be fixed to an old version.

Send a PR?

Current implementation is building a static binary first and sharing the binary on distributed executors. So it is not possible. Java can dynamically send the byte code over the network....