nvda [WIP] Merge Chinese Word Segmentation work

Merges:

#18548
#18735
#18865
#19324

Nov 05 '25 01:11 seanbudd

Will we do any polishing or cleanup work in this PR?

Nov 05 '25 06:11 cary-rowen

@cary-rowen - yes - this is WIP

Nov 05 '25 07:11 seanbudd

What are our main tasks in this PR? Is it just cleaning up the code? Is it possible to include bug fixes? cc @CrazySteve0605 According to OSPP guidelines, is this still considered part of your job responsibilities?

Nov 20 '25 11:11 cary-rowen

cc @CrazySteve0605 According to OSPP guidelines, is this still considered part of your job responsibilities?

Yes. I locally finished fixing remaining issues in #18865, and will make a PR soon, for this functionality to be integrated more properly.

Nov 20 '25 12:11 CrazySteve0605

@cary-rowen @CrazySteve0605 - yes, PRs to this branch that improve the code quality and fix issues would be appreciated. Particularly resolving conflicts

Nov 21 '25 06:11 seanbudd

As we discussed offline @CrazySteve0605

According to feedback from the community, there are still some cases that need to be improved.

I think that when the known to-do items for this PR are solved and merged into the Master, we can provide follow-up polish PR.

Here, just co-pilot provides some comments worth addressing.

Nov 30 '25 00:11 cary-rowen