Error occurred when finalizing GeneratorDataset iterator:

[nomanpust]

After 22 or 25 epochs , I am facing the error: tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated. [[{{node PyFunc}}]] Would you please give me any suggestion??

[jaspereb]

can you include the entire error trace

[nomanpust]

Epoch 00023: saving model to C:/Users/3DSVA-PC-1/RetinanetTutorial/TrainingOutput/snapshots\resnet50_pascal_23.h5

Epoch 00023: ReduceLROnPlateau reducing learning rate to 1.0000000116860975e-08. 100/100 [==============================] - 1872s 19s/step - loss: 2.1196 - classification_loss: 0.2780 - regression_loss: 1.8416 2020-09-25 00:12:21.876334: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated. [[{{node PyFunc}}]]

(retinanet) C:\Users\3DSVA-PC-1\RetinanetTutorial\keras-retinanet> Thank you for your response. Please give me a suggestion.

[jaspereb]

because it occurs at the end of your training epoch, it's likely to be a bug in your validation or test code. You can try setting the --steps argument to be something very small and see if it consistently errors when the validation generator is called a certain number of times.

This is often due to it reaching a certain data point in the validation set which causes it to crash when loaded

[lucysumi]

@jaspereb I'm facing the same issues here. and after reading all the related issues, I still haven't found the solution. Could someone please help? what "exactly" am I supposed to do to fix this problem?

@nomanpust have you found the solution yet?