ibis
ibis copied to clipboard
ibis-project
meta: Release and launch Ibis streaming data backends
Acceptance criteria: Ibis releases and launches its first two streaming backends, Apache Flink and RisingWave, in collaboration with Claypot AI and RisingWave respectively.
Stretch goal: Apache Sedona backend for streaming + geospatial.
TODOs:
- [x] https://github.com/ibis-project/ibis/issues/8036
- [x] https://github.com/ibis-project/ibis/issues/8038
- [x] Apache Flink tutorial + release blog
- [x] https://github.com/ibis-project/ibis/issues/7739
- [ ] RisingWave tutorial + release blog
- [x] Overall "Ibis supports streaming backends" blog
Weekly Update: 2024-01-18
After the sync with Claypot on 01/16/24 these are the PRs that need review for the 8.0.0 release.
Flink To review
(Chloe mentioned that Mehmet's PR should be reviewed first so this is the order):
- test(flink): deep dive on the tests marked for Flink in test_json.py ibis#7908
- feat(flink): implement support for temporal join ibis#7921
- feat(flink): implement array operators ibis#7951
- feat(flink): add new temporal operators ibis#7974
- test(flink): add unit test for window join ibis#7966
Draft PR: conversation ongoing in the comments on whether it's worth it keep this one open
- feat(flink): implement struct field, clean up literal, and adjust timecontext test markers ibis#7997
This is the blog:
Risingwave This is the open PR, Deepyaman mentioned he will help reviewing but this one needs some attention from the team, during the meeting Phillip mentioned he will take a look.
At the moment CI is stuck because it has conflicts, maybe we should give it a nudge.
(copying over last week's update on behalf of Naty)
Weekly Update 2024/01/24
Flink Still open, need to get in before 8.0 release
- https://github.com/ibis-project/ibis/pull/7951 (Looks like needs a final review)
- https://github.com/ibis-project/ibis/pull/7966 (there is a comment pending, and a ci linting job failing)
Merged or closed 🎉
- https://github.com/ibis-project/ibis/pull/7908
- https://github.com/ibis-project/ibis/pull/7974
- https://github.com/ibis-project/ibis/pull/7997
- https://github.com/ibis-project/ibis/pull/7921
Blog PR:
- https://github.com/ibis-project/ibis/pull/7912 (There are few comments to address, but this should get merged and publicized next week )
Tutorial PR:
- https://github.com/ibis-project/ibis/pull/8085 (Draft - WIP)
Risingwave
- https://github.com/ibis-project/ibis/pull/7954 (CI is mostly green, only one failure on a Risingwave job)
Weekly Update
2024/01/31
NOTE: 8.0 Ibis release with streaming support will happen early next week
Flink Still open, need to get in before 8.0 release
- https://github.com/ibis-project/ibis/pull/7951 (Needs a final review - Philip is on it)
Still open but not required for release
- https://github.com/ibis-project/ibis/pull/7966 (there is a comment pending, and a ci linting job failing)
Blog PR:
- https://github.com/ibis-project/ibis/pull/7912 - This one is ready, waiting for 8.0 release to be merged and publicized
Tutorial PR:
- https://github.com/ibis-project/ibis/pull/8085 (Under review)
Risingwave PR was merged 🎉
- https://github.com/ibis-project/ibis/pull/7954
Weekly update
2024/02/07
Flink This PRs were merged and the Flink backend was concluded for the release of Ibis v8.0
- https://github.com/ibis-project/ibis/pull/7951
- https://github.com/ibis-project/ibis/pull/7966
Blog and Tutorial PRs are ready to be merged, waiting for coordinated advertisement push:
- https://github.com/ibis-project/ibis/pull/7912
- https://github.com/ibis-project/ibis/pull/8085
RisingWave
- No update here
@lostmygithubaccount In the main epic comment we have these as requirements. For release and launch.
- RisingWave tutorial + release blog
- Overall "Ibis supports streaming backends" blog
Can we revisit these requirements?
I think having API docs for RisingWave (added here https://github.com/ibis-project/ibis/pull/8286) and the roadmap blog announcement that will have information about streaming should probably be enough for release announcement.
Thoughts?
Weekly update
2024/02/14
- Ibis streaming support was released in v8.0
- Blogs and tutorial on the Flink backend were released
- Coordinated launch and press release were done to promote the streaming feature
Weekly update
2024/02/22
- Overall "Ibis supports streaming backends" blog was covered in the Ibis roadmap blog (https://ibis-project.org/posts/roadmap-2024-h1/)
- Rising wave did a press released about the Ibis-RisingWave backend https://risingwave.com/blog/risingwave-labs-and-voltron-data-collaborate-to-accelerate-stream-processing/
We have pending a RisingWave blog/tutorial on Ibis site.
- [ ] https://github.com/ibis-project/ibis/issues/8424