Problem with training Data

[artur92]

Hi I have the Ground truth and Input, but when I m trying of training the network I get the next error:

File "/Users/.local/lib/python3.6/site-packages/tf_unet-0.1.1-py3.6.egg/tf_unet/image_util.py", line 64, in _process_labels labels[..., 1] = label

ValueError: could not broadcast input array from shape (128,128,3) into shape (128,128)

and I don't how to fix it, is the first time I m working with Machine learning so sorry if is a dumb question. The code that I m using is next one:

data_provider = image_util.ImageDataProvider('/Users/Downloads/Tiff_Files/*.tif')

#setup & training net = unet.Unet(layers=3, features_root=64, channels=3, n_class=3) trainer = unet.Trainer(net) path = trainer.train(data_provider, output_path, training_iters=32, epochs=1)

the problems start when I run trainer.train.

Thanks !

[artur92]

I solved that problem using: data_provider = image_util.ImageDataProvider('/Users/Downloads/Tiff_Files/*.tif') data_provider.n_class =3

but now after 3 iters I got this error: InvalidArgumentError: Nan in summary histogram for: up_convolution_0/activations [[Node: up_convolution_0/activations = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](up_convolution_0/activations/tag, Relu_11)]]

Caused by op 'up_convolution_0/activations', defined at: File "/Users/apple/anaconda3/lib/python3.6/site-packages/spyder/utils/ipython/start_kernel.py", line 245, in main() File "/Users/apple/anaconda3/lib/python3.6/site-packages/spyder/utils/ipython/start_kernel.py", line 241, in main kernel.start() File "/Users/apple/anaconda3/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 477, in start ioloop.IOLoop.instance().start() File "/Users/apple/anaconda3/lib/python3.6/site-packages/zmq/eventloop/ioloop.py", line 177, in start super(ZMQIOLoop, self).start() File "/Users/apple/anaconda3/lib/python3.6/site-packages/tornado/ioloop.py", line 888, in start handler_func(fd_obj, events) File "/Users/apple/anaconda3/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper return fn(*args, **kwargs) File "/Users/apple/anaconda3/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events self._handle_recv() File "/Users/apple/anaconda3/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv self._run_callback(callback, msg) File "/Users/apple/anaconda3/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 414, in run_callback callback(*args, **kwargs) File "/Users/apple/anaconda3/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper return fn(*args, **kwargs) File "/Users/apple/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher return self.dispatch_shell(stream, msg) File "/Users/apple/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 235, in dispatch_shell handler(stream, idents, msg) File "/Users/apple/anaconda3/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 399, in execute_request user_expressions, allow_stdin) File "/Users/apple/anaconda3/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 196, in do_execute res = shell.run_cell(code, store_history=store_history, silent=silent) File "/Users/apple/anaconda3/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 533, in run_cell return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs) File "/Users/apple/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2698, in run_cell interactivity=interactivity, compiler=compiler, result=result) File "/Users/apple/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2808, in run_ast_nodes if self.run_code(code, result): File "/Users/apple/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2862, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in runfile('/Users/apple/Documents/GitHub/Melanoma_Segmentation_Classification/Unet', wdir='/Users/apple/Documents/GitHub/Melanoma_Segmentation_Classification') File "/Users/apple/anaconda3/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 710, in runfile execfile(filename, namespace) File "/Users/apple/anaconda3/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 101, in execfile exec(compile(f.read(), filename, 'exec'), namespace) File "/Users/apple/Documents/GitHub/Melanoma_Segmentation_Classification/Unet", line 70, in net = unet.Unet(layers=3, features_root=64, channels=3, n_class=3) File "/Users/apple/.local/lib/python3.6/site-packages/tf_unet-0.1.1-py3.6.egg/tf_unet/unet.py", line 189, in init logits, self.variables, self.offset = create_conv_net(self.x, self.keep_prob, channels, n_class, **kwargs) File "/Users/apple/.local/lib/python3.6/site-packages/tf_unet-0.1.1-py3.6.egg/tf_unet/unet.py", line 154, in create_conv_net tf.summary.histogram("up_convolution%s"%k + '/activations', up_h_convs[k]) File "/Users/apple/anaconda3/lib/python3.6/site-packages/tensorflow/python/summary/summary.py", line 196, in histogram tag=tag, values=values, name=scope) File "/Users/apple/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_logging_ops.py", line 282, in _histogram_summary "HistogramSummary", tag=tag, values=values, name=name) File "/Users/apple/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/Users/apple/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3290, in create_op op_def=op_def) File "/Users/apple/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1654, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Nan in summary histogram for: up_convolution_0/activations [[Node: up_convolution_0/activations = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](up_convolution_0/activations/tag, Relu_11)]]

any idea ? Thanks

[artur92]

I was testing this

data_provider = image_util.ImageDataProvider('/Users/apple/Downloads/Tiff_Files/*.tif') data_provider.n_class = 3

net = unet.Unet(layers=3,features_root=64,channels=3,n_class=3)

trainer = unet.Trainer(net, optimizer="adam") path = output_path

path = trainer.train(data_provider, path, training_iters=32, epochs=1, dropout=0.5, display_step=10)

but I get this RunTime Warning without finished the 32 iters

/Users/apple/.local/lib/python3.6/site-packages/tf_unet-0.1.1-py3.6.egg/tf_unet/util.py:74: RuntimeWarning: invalid value encountered in true_divide img /= np.amax(img) 2018-04-10 01:07:55,977 Optimization Finished!

Im not sure what is going on.

Thanks !

[neurmch]

I have the same problem with you , have you solved the last Warning? Can you tell me how you solve it ?Thank you!

[neurmch]

my predict images is all black , I don't know why.

[artur92]

No I m having the same problem if you find the solution please let me know

[jakeret]

I could be that the training is not converging. How does the Tensorboard look like? Is the loss decreasing?