XL2248
ValueError: Cannot create a tensor proto whose content is larger than 2GB.
Hello, when training with the DailyDialog dataset, the problem is:ValueError: Cannot create a tensor proto whose content is larger than 2GB. Through analysis, it is found that the problem is in this line of code: A = tf.convert_to_tensor(value=A, dtype=tf.float32).
I want to ask how you handled this line of code when you were training at that time? I hope to get your reply. thank you!
Hi， As far as I know, "...2GB" is a hard limitation and you may reduce the size of A to less than 2GB. In my memory, I don't have this issue in this experiment. Is there some difference when producing A?
Firstly, thank you for your reply. The generation of A is the same, but because of the problem of data volume, the dimensions of the vectors are different. If the data set is larger, this problem will occur. Can you conveniently share the DailyDialog dataset you used in the experiment? I want to see if there is a problem with the dataset I downloaded, and this problem occurs after the dataset is processed.