GeoddHQ comments

Results 5 comments of


                                            GeoddHQ

Under the main branch, stress testing the in-flight Triton Server with multiple threads can result in the Triton Server getting stuck.

This happens when I quantize the model with TP 8 or multi GPU, Then when you run launch_triton_server.py the CPU is at 100% and hangs. If I run on TP...

Under the main branch, stress testing the in-flight Triton Server with multiple threads can result in the Triton Server getting stuck.

@akhoroshev thanks for the suggestion, still it hangs ``` I0327 13:12:22.342290 146 grpc_server.cc:2558] "Started GRPCInferenceService at 0.0.0.0:8001" I0327 13:12:22.342523 146 http_server.cc:4725] "Started HTTPService at 0.0.0.0:8000" I0327 13:12:22.542293 146 http_server.cc:358] "Started...

Why tensorrt_llm_bls backend doesn't support speculative decoding streaming or bsz > 1?

@meowcoder22 did you find a fix ?

Add ZFS.md

@mogoll92 @jcstein @kinrokinro After many days of testing, we finally found the solution. I’ve shared all the key learnings here: [Celestia Testnet - Bridge DA Node ZFS Optimizations](https://networks.geodd.io/docs/celestia/bridge-da-node/zfs-storage-optional#zfs-optimizations-for-fast-sync-). Initially, we...

Add ZFS.md

I agree that the sync should be enabled after its completed. ### Here are some stats before optimization ### After Optimization ### Used space