Error when running RBERT In Tensorflow 1.11.0: "Error in py_call_impl(callable, dots$args, dots$keywords)"

[Abbiegalie]

Hi,

I am trying to run RBERT in tensorflow on small dataset. I have installed Tensorflow using the miniconda environment. Below is the code which throws the error:

Sys.setenv(RETICULATE_PYTHON = "/Users/applemacbookpro/opt/miniconda3/envs/tensorflowa/bin/python")

#Make virtual environment in anaconda

reticulate::conda_list()[[1]][8] %>% 
  reticulate::use_condaenv(required = TRUE)


#Load the libraries
library(keras)
library(tidyverse)
library(stringr)
library(tidytext)
library(caret)
library(dplyr)
library(tm)
library(RBERT)
library(tensorflow)
library(reticulate)

#Install RBERT

devtools::install("/Users/applemacbookpro/Downloads/RBERT")
        
#Initiate BERT
BERT_PRETRAINED_DIR <- RBERT::download_BERT_checkpoint(model = "bert_base_uncased")


#Extract tokenized words from agency trainset
BERT_feats <- extract_features(
  examples = agency_trainset$agency,
  ckpt_dir = BERT_PRETRAINED_DIR,
  layer_indexes = 1:12,
)

Error in py_call_impl(callable, dots$args, dots$keywords) : RuntimeError: Evaluation error: ValueError: Tried to convert 'size' to a tensor and failed. Error: Cannot convert a partially known TensorShape to a Tensor: (128, ?).

Traceback:
stop(structure(list(message = "RuntimeError: Evaluation error: ValueError: Tried to convert 'size' to a tensor and failed. Error: Cannot convert a partially known TensorShape to a Tensor: (128, ?).", 
call = py_call_impl(callable, dots$args, dots$keywords), 
cppstack = structure(list(file = "", line = -1L, stack = c("1 reticulate.so 0x000000010773d3de _ZN4Rcpp9exceptionC2EPKcb + 222", 
"2 reticulate.so 0x0000000107746245 _ZN4Rcpp4stopERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE + 53", ...
13.
python_function at call.py#21
12.
fn at <string>#4
11.
_call_model_fn at tpu_estimator.py#1524
10.
call_without_tpu at tpu_estimator.py#1250
9.
_model_fn at tpu_estimator.py#2470
8.
_call_model_fn at estimator.py#1169
7.
_call_model_fn at tpu_estimator.py#2186
6.
predict at estimator.py#551
5.
predict at tpu_estimator.py#2431
4.
raise_errors at error_handling.py#128
3.
predict at tpu_estimator.py#2437
2.
result_iterator$`next`()
1.
extract_features(examples = agency_trainset$agency, ckpt_dir = BERT_PRETRAINED_DIR, 
layer_indexes = 1:12, )

[jonathanbratt]

Thanks for the note! My best guess is that it's a python version issue and/or a TensorFlow version issue. Unfortunately, RBERT has been in an update limbo between TF versions for a long time. Because of the headaches associated with dealing with python/TF in R, we have focused our efforts on reimplementing RBERT functionality using the new(ish) {torch} R-package. We've found that getting torch working is much easier for most people than getting python/TF working. You can check out our {torchtransformers} package here: https://github.com/macmillancontentscience/torchtransformers

The torchtransformers package already enables the construction, loading, fine-tuning, and execution of pretrained BERT models. It's not yet as user-friendly as RBERT (the output isn't tidied up, and you have to do more of the input processing), but we're working on that.