`RecursionError: maximum recursion depth exceeded while calling a Python object` when inferencing with long input

[Ja1Zhou]

Hi, I ran across this issue during inference

Exception in ModelRpcClient:
Traceback (most recent call last):
  File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/sglang/srt/managers/router/model_rpc.py", line 168, in exposed_step
    self.forward_step()
  File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/sglang/srt/managers/router/model_rpc.py", line 195, in forward_step
    self.forward_decode_batch(self.running_batch)
  File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/sglang/srt/managers/router/model_rpc.py", line 460, in forward_decode_batch
    self.handle_finished_requests(batch)
  File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/sglang/srt/managers/router/model_rpc.py", line 528, in handle_finished_requests
    prefix_len = self.tree_cache.insert(
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/sglang/srt/managers/router/radix_cache.py", line 61, in insert
    return self._insert_helper(self.root_node, key, value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/sglang/srt/managers/router/radix_cache.py", line 157, in _insert_helper
    return prefix_len + self._insert_helper(child, key, value)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/sglang/srt/managers/router/radix_cache.py", line 157, in _insert_helper
    return prefix_len + self._insert_helper(child, key, value)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/sglang/srt/managers/router/radix_cache.py", line 157, in _insert_helper
    return prefix_len + self._insert_helper(child, key, value)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 958 more times]
  File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/sglang/srt/managers/router/radix_cache.py", line 166, in _insert_helper
    new_node = TreeNode()
               ^^^^^^^^^^
  File "/User/jay/miniconda3/envs/sglang/lib/python3.11/site-packages/sglang/srt/managers/router/radix_cache.py", line 12, in __init__
    self.children = defaultdict(TreeNode)
                    ^^^^^^^^^^^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded while calling a Python object

Would it be possible to implement this logic without recursion? @merrymercy