research

Low QPS with momentary traffic surges cause significant increases in inference TP99 latency.

1

**Description** Low QPS with momentary traffic surges cause significant increases in inference TP99 latency. **Triton Information** 23.06  

一张T4卡跑的模型，

使用self.model = torchacc.accelerate(model)加速模型训练的时，GPU利用率极低，接近于0

1

``` class Trainer: def __init__(self, global_rank, gpu_id: int, trainer_config: TrainerConfig, model: RecNet, optimizer, world_size: int, data_cfg: DataConfig): self.global_rank = global_rank self.config = trainer_config self.world_size = world_size self.dataloader = Data(data_cfg) self.epochs_run...

xlarun --nproc_per_node=8 YOUR_MODEL.py

6

xlarun: command not found, I used the container you provided, but the command is not found.

Has there been an improvement in single-GPU training speed?

1

@anw90

[Question] <title>When I serialize this object, enough space is allocated, but it still throws a java.lang.OutOfMemoryError: Java heap space

10

``` public class FlatStorage implements Serializable { private MemoryBuffer buf; private Map featureMetadata; public FlatStorage(int bufferSize) { this.buf = MemoryUtils.buffer(bufferSize); this.featureMetadata = new HashMap(); } public void addFeature(String name, int...

[Question] <title>How much faster is Fury compared to Protobuf, and does the measurement include the time taken for Protobuf construction?

1

### Question How much faster is Fury compared to Protobuf, and does the measurement include the time taken for Protobuf construction?

[Question] <title>How does Fury perform in terms of speed and size when serializing and deserializing Java arrays?

2

### Question How does Fury perform in terms of speed and size when serializing and deserializing Java arrays?