BigDL-2.x icon indicating copy to clipboard operation
BigDL-2.x copied to clipboard

intel tensorflow 1.15.0up3 testing

Open yangw1234 opened this issue 3 years ago • 6 comments

  • [ ] bf16 verification
  • [x] make sure oneDNN ops can be turned off
  • [ ] inception v1 example
  • [x] other examples, ut
  • [ ] wnd memory leak problem

yangw1234 avatar Aug 09 '21 07:08 yangw1234

@sgwhat

yangw1234 avatar Aug 09 '21 07:08 yangw1234

@sgwhat

Roger that

sgwhat avatar Aug 09 '21 07:08 sgwhat

In the tf2/resnet training data part, we'd better change the URL of instruction. The original tensorflow/model repository's slim bazel BUILD file is broken and not working, this repository updated BUILD and it works . @yangw1234

sgwhat avatar Aug 10 '21 02:08 sgwhat

In the tf2/resnet training data part, we'd better change the URL of instruction. The original tensorflow/model repository's slim bazel BUILD file is broken and not working, this repository updated BUILD and it works . @yangw1234

Let's use dummpy data first

yangw1234 avatar Aug 10 '21 02:08 yangw1234

  • [ ] bf16 verification
  • [x] make sure oneDNN ops can be turned off
  • [x] inception v1 example
  • [x] other examples, ut
  • [x] wnd memory leak problem

sgwhat avatar Aug 12 '21 08:08 sgwhat

Continue adding more results of other examples

  • v1.15.0up3 Tensorflow
Task Name Loss Top1Accuracy Running Time(s)
Inception v1(Part) 5.744978 0.0025575447 1258.7487390041351
Basic_text_classification 4.087298393249512 0.8654000163078308 151.16134071350098
Transfer_learning 2.854679584503174 0.9340000152587891 355.7099766731262
Image_segmentation 0.07223675400018692 N/A 269.35604453086853
Keras_dataset 0.114413782954216 0.9639999866485596 32.79020833969116
Keras_ndarray 0.09849555790424347 0.9689000248908997 55.92670226097107
  • v1.15.0up3 Disable MKL-DNN Tensorflow
Task Name Loss Top1Accuracy Running Time(s)
Inception v1(Part) 4.9309444 0.0051150895 1634.7301709651947
Basic_text_classification 4.24573278427124 0.8626999855041504 148.5364646911621
Transfer_learning 2.6535277366638184 0.9449999928474426 446.9547288417816
Image_segmentation 0.07158134132623672 N/A 480.3870759010315
Keras_dataset 0.0991317480802536 0.9692000150680542 34.12212657928467
Keras_ndarray 0.09951204806566238 0.9718000292778015 58.7927188873291
  • Original Version
Task Name Loss Top1Accuracy Running Time(s)
Inception V1(Part) 5.7137446 0.0076726344 1312.0718114376068
Basic_Text_Classification 4.267351150512695 0.8640000224113464 171.9112410545349
Transfer_Learning 2.574239730834961 0.9629999995231628 475.75219345092773
Image_segmentation 0.07637828588485718 N/A 305.888587474823
Keras_dataset 0.1009390577673912 0.9696999788284302 34.34745478630066
Keras_ndarray 0.09711404144763947 0.9707000255584717 60.50141477584839
  • Performance(Speed) Comparison
Task Name Original Version Running Time(s) v1.15.0up3 Running Time(s) v1.15.0up3 Disable MKL-DNN Running Time(s) Speed Up (v1.15.0up3&Original)
Inception V1(Part) 1312.0718114376068 1258.7487390041351 1634.7301709651947 4.06%
Basic_Text_Classification 171.9112410545349 151.16134071350098 148.5364646911621 12.07%
Transfer_Learning 475.75219345092773 355.7099766731262 446.9547288417816 25.23%
Image_segmentation 305.888587474823 269.35604453086853 480.3870759010315 11.94%
Keras_dataset 34.34745478630066 32.79020833969116 34.12212657928467 4.53%
Keras_ndarray 60.50141477584839 55.92670226097107 58.7927188873291 7.56%
  • Performance(Accuracy) Comparison
Task Name Original Version Running Time(s) v1.15.0up3 Running Time(s) v1.15.0up3 Disable MKL-DNN Running Time(s) Optimization (v1.15.0up3&Original)
Inception V1(Part) 0.0076726344 0.0025575447 0.0051150895
Basic_Text_Classification 0.8640000224113464 0.8654000163078308 0.8626999855041504
Transfer_Learning 0.9629999995231628 0.9340000152587891 0.9449999928474426
Image_segmentation N/A N/A N/A N/A
Keras_dataset 0.9696999788284302 0.9639999866485596 0.9692000150680542
Keras_ndarray 0.09711404144763947 0.9689000248908997 0.9718000292778015

sgwhat avatar Aug 17 '21 09:08 sgwhat