cehr-bert icon indicating copy to clipboard operation
cehr-bert copied to clipboard

Published 20 hours ago • verified publisher icon cumc-dbmi

Switch from our own implementation of MultiHeadAttention to the one created by tensorflow

Open ChaoPang opened this issue 1 year ago • 0 comments

  • Upload Tensorflow to 2.12 so we can use TensorFlow's version of MultiHeadAttention
  • Replace our own implementation of MultiHeadAttention with the Tensorflow version of MultiHeadAttention
  • Toggle mask at https://github.com/cumc-dbmi/cehr-bert/blob/8862fd6a1f026839d301625fb30a1ba7e19a4672/data_generators/learning_objective.py#L281. Because mask=1 indicates attention whereas mask=0 represents no attention

ChaoPang avatar May 30 '23 00:05 ChaoPang