Basic_CNNs_TensorFlow2 icon indicating copy to clipboard operation
Basic_CNNs_TensorFlow2 copied to clipboard

A little problem about dataset

Open TIMOLEEGO opened this issue 4 years ago • 1 comments

I'm a newcomer to Machine Learning. It's my pleasure to run your code. When I run split_dataset.py, no data is copied into folders train and test. I copy the data from folder original_dataset into folders test and train, and run train.py will report an error: `Traceback (most recent call last): File "C:/Python/Tensorflow/TensorFlow2.0_ResNet-master/train.py", line 33, in train_dataset, valid_dataset, test_dataset, train_count, valid_count, test_count = generate_datasets() File "C:\Python\Tensorflow\TensorFlow2.0_ResNet-master\prepare_data.py", line 47, in generate_datasets train_dataset, train_count = get_dataset(dataset_root_dir=config.train_dir) File "C:\Python\Tensorflow\TensorFlow2.0_ResNet-master\prepare_data.py", line 38, in get_dataset image_dataset = tf.data.Dataset.from_tensor_slices(all_image_path).map(load_and_preprocess_image) File "C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 1588, in map return MapDataset(self, map_func, preserve_cardinality=True) File "C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 3888, in init use_legacy_function=use_legacy_function) File "C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 3147, in init self._function = wrapper_fn._get_concrete_function_internal() File "C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\eager\function.py", line 2395, in _get_concrete_function_internal *args, **kwargs) File "C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\eager\function.py", line 2389, in _get_concrete_function_internal_garbage_collected graph_function, _, _ = self._maybe_define_function(args, kwargs) File "C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\eager\function.py", line 2703, in _maybe_define_function graph_function = self._create_graph_function(args, kwargs) File "C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\eager\function.py", line 2593, in _create_graph_function capture_by_value=self._capture_by_value), File "C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\framework\func_graph.py", line 978, in func_graph_from_py_func func_outputs = python_func(*func_args, **func_kwargs) File "C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 3140, in wrapper_fn ret = _wrapper_helper(*args) File "C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 3082, in _wrapper_helper ret = autograph.tf_convert(func, ag_ctx)(*nested_args) File "C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\autograph\impl\api.py", line 237, in wrapper raise e.ag_error_metadata.to_exception(e) TypeError: in converted code:

C:\Python\Tensorflow\TensorFlow2.0_ResNet-master\prepare_data.py:9 load_and_preprocess_image  *
    img_raw = tf.io.read_file(img_path)
C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\ops\gen_io_ops.py:568 read_file
    "ReadFile", filename=filename, name=name)
C:\Python\Tensorflow\test\venv\lib\site-packages\tensorflow_core\python\framework\op_def_library.py:491 _apply_op_helper
    (prefix, dtypes.as_dtype(input_arg.type).name))

TypeError: Input 'filename' of 'ReadFile' Op has type float32 that does not match expected type of string.

Process finished with exit code 1 ` I'm also a newcomer to tensorflow. How can I solve this problem?

TIMOLEEGO avatar Mar 02 '20 13:03 TIMOLEEGO

You don't have to use the data pipeline to process the data, you can use tensorflow_datasets or keras.datasets to load the input data, unless the data you are training on aren't included in these datasets, in which case you can still use tf.data.Dataset to process the data.

keras.datasets

tensorflow_datasets

tf.data.Dataset

I was trying to write some tutorial on tensorflow, the first note I made is about the tensorflow_datasets. If you need, you can check it out as well.

Davidxswang avatar Jul 14 '20 07:07 Davidxswang