Shiqing Fan

icon indicating a website link https://fanshiqing.github.io/ icon indicating an email [email protected]

icon company NVIDIA icon location Hangzhou, Zhejiang icon location NVIDIA, Senior Performance Architect, Full-stack LLM Training Optimization.

Results 2 issues of Shiqing Fan

Abnormal memory allocation of transformer with GPipe on V100-16GB

1 comment

**Codebase Version**: - Lingvo: https://github.com/tensorflow/lingvo/tree/tf1.15 - Docker: build from https://github.com/tensorflow/lingvo/blob/tf1.15/docker/dev.dockerfile **Hardward config** - V100-16GB x 8 with NVLink **Problem Description** For default hyper-parameter as described [here](https://github.com/tensorflow/lingvo/blob/tf1.15/lingvo/tasks/lm/params/one_billion_wds.py#L174-L198), direct use of these...

SocketCaffeNet UT should be enhanced

4 comment

After carefully check of the codebase, it seems that currently the [SocketCaffeNet related UT](https://github.com/yahoo/CaffeOnSpark/blob/master/caffe-distri/src/test/java/com/yahoo/ml/jcaffe/CaffeNetTest.java#L61) is not so make sense. I'm curious that how did you ensure the correctness of your...