GPU on Notebooks running on Big Sur w/ tensorflow-macos / tensorflow-metal

[dbl001]

I was evaluating whether these notebooks work with my AMD Radeon Pro 5700 XT. I have been able to get Keras models to use the GPU, however, the 'chapter07_working_with_keras' and the 'chapter11_part04_sequence_to_sequence' notebooks do not appear to be using the GPU. I installed tensorflow-macos and tensorflow-metal with these instructions:

https://developer.apple.com/metal/tensorflow-plugin/

I had to create the virtual environment with Apple's python 3.8.2. Anaconda's python 3.8.5 didn't work. E.g. /Library/Developer/CommandLineTools/usr/bin/python3 -m venv tensorflow-metal

2021-07-15 12:03:01.235594: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at partitioned_function_ops.cc:114 : Invalid argument: No OpKernel was registered to support Op 'CudnnRNNV3' used by {{node cond_41/then/_0/cond/CudnnRNNV3}} with these attrs: [num_proj=0, time_major=false, dropout=0, seed=0, T=DT_FLOAT, input_mode="linear_input", direction="unidirectional", rnn_mode="gru", is_training=true, seed2=0]
Registered devices: [CPU, GPU]
Registered kernels:
  <no registered kernels>

	 [[cond_41/then/_0/cond/CudnnRNNV3]]

% pip list
Package                  Version
------------------------ -------------------
absl-py                  0.12.0
appnope                  0.1.2
astunparse               1.6.3
attrs                    21.2.0
backcall                 0.2.0
cachetools               4.2.2
certifi                  2021.5.30
charset-normalizer       2.0.1
Cython                   0.29.24
debugpy                  1.3.0
decorator                5.0.9
dill                     0.3.4
flatbuffers              1.12
future                   0.18.2
gast                     0.4.0
google-auth              1.32.1
google-auth-oauthlib     0.4.4
google-pasta             0.2.0
googleapis-common-protos 1.53.0
grpcio                   1.34.1
h5py                     3.1.0
idna                     3.2
importlib-resources      5.2.0
ipykernel                6.0.1
ipython                  7.25.0
ipython-genutils         0.2.0
jedi                     0.18.0
jupyter-client           6.1.12
jupyter-core             4.7.1
keras-nightly            2.5.0.dev2021032900
Keras-Preprocessing      1.1.2
Markdown                 3.3.4
matplotlib-inline        0.1.2
numpy                    1.19.5
oauthlib                 3.1.1
opt-einsum               3.3.0
parso                    0.8.2
pexpect                  4.8.0
pickleshare              0.7.5
pip                      21.1.3
promise                  2.3
prompt-toolkit           3.0.19
protobuf                 3.17.3
ptyprocess               0.7.0
pyasn1                   0.4.8
pyasn1-modules           0.2.8
pybind11                 2.6.2
Pygments                 2.9.0
python-dateutil          2.8.2
pyzmq                    22.1.0
requests                 2.26.0
requests-oauthlib        1.3.0
rsa                      4.7.2
setuptools               41.2.0
six                      1.15.0
tensorboard              2.5.0
tensorboard-data-server  0.6.1
tensorboard-plugin-wit   1.8.0
tensorflow-datasets      4.3.0
tensorflow-estimator     2.5.0
tensorflow-macos         2.5.0
tensorflow-metadata      1.1.0
tensorflow-metal         0.1.1
termcolor                1.1.0
tornado                  6.1
tqdm                     4.61.2
traitlets                5.0.5
typing-extensions        3.7.4.3
urllib3                  1.26.6
wcwidth                  0.2.5
Werkzeug                 2.0.1
wheel                    0.36.2
wrapt                    1.12.1
zipp                     3.5.0

This Keras code successfully runs on the GPU:

import tensorflow_datasets as tfds
import tensorflow as tf

tf.compat.v1.enable_v2_behavior()

from tensorflow.python.framework.ops import disable_eager_execution
disable_eager_execution()


(ds_train, ds_test), ds_info = tfds.load(
    'mnist',
    split=['train', 'test'],
    shuffle_files=True,
    as_supervised=True,
    with_info=True,
)

def normalize_img(image, label):
  """Normalizes images: `uint8` -> `float32`."""
  return tf.cast(image, tf.float32) / 255., label

batch_size = 128

ds_train = ds_train.map(
    normalize_img, num_parallel_calls=tf.data.experimental.AUTOTUNE)
ds_train = ds_train.cache()
ds_train = ds_train.shuffle(ds_info.splits['train'].num_examples)
ds_train = ds_train.batch(batch_size)
ds_train = ds_train.prefetch(tf.data.experimental.AUTOTUNE)


ds_test = ds_test.map(
    normalize_img, num_parallel_calls=tf.data.experimental.AUTOTUNE)
ds_test = ds_test.batch(batch_size)
ds_test = ds_test.cache()
ds_test = ds_test.prefetch(tf.data.experimental.AUTOTUNE)


model = tf.keras.models.Sequential([
  tf.keras.layers.Conv2D(32, kernel_size=(3, 3),
                 activation='relu'),
  tf.keras.layers.Conv2D(64, kernel_size=(3, 3),
                 activation='relu'),
  tf.keras.layers.MaxPooling2D(pool_size=(2, 2)),
#   tf.keras.layers.Dropout(0.25),
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(128, activation='relu'),
#   tf.keras.layers.Dropout(0.5),
  tf.keras.layers.Dense(10, activation='softmax')
])
model.compile(
    loss='sparse_categorical_crossentropy',
    optimizer=tf.keras.optimizers.Adam(0.001),
    metrics=['accuracy'],
)

model.fit(
    ds_train,
    epochs=12,
    validation_data=ds_test,
)