LIU ZHE YOU

icon indicating a website link blog.zhu424.dev

icon company NCKU CSIE icon location Taiwan , Tainan icon location @apache Airflow Committer | Junior from Taiwan :taiwan: passionate about contributing to OSS. Backend Intern @Appier

Results 63 comments of LIU ZHE YOU

WIP: [Resolve OOM When Reading Large Logs in Webserver] Refactor to Use K-Way Merge for Log Streams Instead of Sorting Entire Log Records

Just rebase to latest main, nothing update.

WIP: [Resolve OOM When Reading Large Logs in Webserver] Refactor to Use K-Way Merge for Log Streams Instead of Sorting Entire Log Records

Hi @potiuk, may I ask whether this PR is considered a refactor for 3.0 or 2.10? I saw the 3.0 feature freeze mentioned in the dev list, so I’m not...

WIP: [Resolve OOM When Reading Large Logs in Webserver] Refactor to Use K-Way Merge for Log Streams Instead of Sorting Entire Log Records

Hi @dstandish @ashb, hope you're doing well! Could you please review this PR when you have some time? Thanks! 🙏

WIP: [Resolve OOM When Reading Large Logs in Webserver] Refactor to Use K-Way Merge for Log Streams Instead of Sorting Entire Log Records

> I'm still half way finish reviewing it. Left a few nitpicks, but the PR is great to be honest. Thanks, @Lee-W, for reviewing! I’ve just resolved those nits. The...

WIP: [Resolve OOM When Reading Large Logs in Webserver] Refactor to Use K-Way Merge for Log Streams Instead of Sorting Entire Log Records

Since the TaskHandler logger being migrate to structlog, I will create another PR for the refactor instead of resolve conflict on this one( too much code change and conflict on...

WIP: [Resolve OOM When Reading Large Logs in Webserver] Refactor to Use K-Way Merge for Log Streams Instead of Sorting Entire Log Records

Close this PR since it’s superseded by: - #49470 (targets `3.0+`) - #45914 (targets `2.11`)

Resolve OOM When Reading Large Logs in Webserver

There is still an issue with the `compat 3.0.0` tests for `ElasticsearchTaskHandler` and `OpensearchTaskHandler`, as they directly implement the `FileTaskHandler._read` method. This refactor introduces a new **streaming** return type for...

Resolve OOM When Reading Large Logs in Webserver

> I am afraid we will have to make it backwards-compatible in some way. I don't think we are going to yank 3.0.0, so theorethically (and many people do that)...

Resolve OOM When Reading Large Logs in Webserver

The PR to resolve OOM when reading large logs in the webserver, targeting the 3.0+ branch, is ready for review. I’d really appreciate your feedback.

Resolve OOM When Reading Large Logs in Webserver

Rebase to latest main to resolve conflict, wait for review.