Wojciech Uss comments

Repositories
Issues
Comments

Results 13 comments of


                                            Wojciech Uss

Optimize inference performance of ERNIE INT8 on CPU

Here come our latest performance results for Ernie FP32 and INT8 runs. The tests were run with affinity settings ``` export KMP_AFFINITY=granularity=fine,compact,1,0 export KMP_BLOCKTIME=1 ``` on CLX 6248. With the...

Optimize inference performance of ERNIE INT8 on CPU

We have finally resolved the accuracy issue for Ernie INT8 run. Background: FC operators in the Ernie model have 2-dimensional weights and receive 3-dimensional inputs. For the MKL-DNN inner product...

Optimize inference performance of ERNIE INT8 on CPU

With the fix for FC INT8 (PR https://github.com/PaddlePaddle/Paddle/pull/22404, branch `Sand3r-:mgallus/3d-fc-acc`) we got the following results for Ernie on a CLX 6248 machine: | | accuracy | latency (ms) 1 thread...