keras-preprocessing
keras-preprocessing copied to clipboard
keras-team
Flow_from_dataframe - 'DataFrameIterator' object has no attribute 'shape'
Hey,
I have been trying to get flow_from_dataframe to work for quite some time now, I'm using Keras 2.2.4 with preprocessing 1.0.9 and Tensorflow 1.13. I'm trying to do a measurement of how much the split between validation and training data affects accuracy and loss. That is why I want to run my CNN hundreds of times with input data being randomly split in validation and training. To have more control how the data is loaded I'm using flow_from_dataframe. I'm loading the data in a dataframe with absolute paths(you can see in the log how it looks). My data are images that either show a specific building(1) or not(0), hence I'm using binary as class_mode. I have my images in a folder called "bilder" and a subfolder containing the building called ("zwinger") and another subfolder is containing images that do not show the building, the folder is called("nicht_zwinger").
My python script: https://gist.github.com/PowerOfCreation/9595ed1907b9baeb40322fc4a314439c The output: https://gist.github.com/PowerOfCreation/70ff9ba040946ba20d5de2aeb0769613
I found many other issues about flow_from_dataframe, so I'm not sure if this is related.
I figured out that the reason that I'm getting this error, is because I'm using the tensorflow.keras version and not the "standalone" Keras installation. Is there any way of getting this keras_preprocessing library to work with tensorflow.keras? I'm working on an HPC cluster and there is only Tensorflow available and I'm advised to use tensforflow.keras.
It is working with Keras 2.2.4, but not when I use Tensorflow 1.13 with the integrated Keras(tensorflow.keras) 2.2.4. Is there any way to get it to work with tensorflow.keras? I cannot install Keras on the machine I'm working on it, but I can use the integrated Keras version inside of Tensorflow.
I think I have exactly the same problem with but with tensorflow 2.0 and keras-preprocessing 1.1.0
But I think there is an easy workaround: just wrap the DataFrameIterator in a python generator like this:
def wrapper_gen(x): yield from x
