addons
addons copied to clipboard
Published
20 hours ago •
tensorflow
tensorflow
How to speed up Lookahead in first epoch.
System information
- OS Platform and Distribution : Linux Ubuntu 20.04
- TensorFlow version and how it was installed (source or binary): tf==2.8.2, source
- TensorFlow-Addons version and how it was installed (source or binary): tfa==0.17.0, source
- Python version: 3.8
- Is GPU used? (yes/no): yes
Describe the bug
When use tfa - lookahead optimizer, it need to cost a long time to start the first training step. During this time, only one CPU is used. GPUs are only allocated memory but the utilization. is zero. Is there any parameter or function to speed up this situation?
Other info / logs In my case . The training/validation dataset is sending by tf.Data.dataset. Model build in tf.keras.models. When model compile in lookahead + adam ,it cost 6073s to finish first epoch .The second epoch only use 769s. When model cmopile in adam only, it cost 1019s to finish first. The second epoch cost 521s.
