How to use it?

[red010182]

I just clone this repo and mkdir some neccessary folders and downloaed vgg16_weights.npz, then run python matting_deconv.py but got these errors:

$ python matting_deconv.py 
range_size is 0

...(tensorflow CPU instruction warnings)

device 0 with properties: 
name: GeForce GTX 1080
major: 6 minor: 1 memoryClockRate (GHz) 1.835
pciBusID 0000:01:00.0
Total memory: 7.92GiB
Free memory: 6.91GiB
2017-09-11 03:35:32.820237: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0 
2017-09-11 03:35:32.820244: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   Y 
2017-09-11 03:35:32.820253: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
Restoring pretrained model...
Traceback (most recent call last):
  File "matting_deconv.py", line 388, in <module>
    saver.restore(sess,tf.train.latest_checkpoint('./model'))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1548, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 789, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 997, in _run
    feed_dict_string, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1132, in _do_run
    target_list, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: Unable to get element from the feed as bytes.

[Joker316701882]

Thank you. I added some images in "data_structure" to show the file structure of training data. note: 'eps' means the images which you can composite with alpha to get FG (i.e. where FGs come from)

Basically:

1. Resized the all eps images and corresponding alpha(or called alpha, matte...) so that the longest side of any eps and alpha images equals to 640 pixel(keep the proportion of width and height).
2. Resized all bg images so that the shortest side of any bg images equals to 1280 pixel(also keep the proportion of width and height. And here remember the the number of bg ->"N_bg=N_fg *100" )
3. Padding eps images and alpha images with 0, make the eps and alpha the same size as corresponding bg image. ('corresponding' means same folder path and same image name.)

Command for running: python matting_unpool.py

[hector-sab]

In case anyone is intereste, I have made some code to create the database. Its main porpuse is to generate it from SURREAL database (Synthetic humans). But there's one class that can do the trick for others sources. https://github.com/hector-sab/DIM_DataCreation .

[bhack]

@hector-sab Nice.. have you tried to train on this data?

[bitname]

@Joker316701882 你好，打扰你了，一直关注这篇论文和你共享的代码，非常感谢你的分享（包括代码、博客以及你的回答），让我渐渐理清了论文脉络，在实验过程中，遇到了一些问题，所以向你咨询一下。

1. 你有没有执行过作者的合成代码？按照作者提供的数据集和合成代码，我合成的每张图片都是1-2M，整个4万多张得有70G-80G吧，合成过程比较慢，3个小时4000张图片，这种速度得30个小时能合成完。这是正常的吗？
2. 作者合成图片的方式和你合成图片的方式不一样。作者的方式是1个alpha，1个fg，100个bg，100个尺寸一样的RGB；而你的方式是由1个alpha，1个fg，100个bg得到100个alpha，100个fg，100个RGB，尺寸都不一样；但每个alpha，fg，RGB，bg的尺寸都是一样的。我感觉你的方式泛化性能是不是更好呢？（作者采用crop的方式提高泛化性能），不知你有没有比较这两种方式，哪种效果更好？

[Joker316701882]

@bitname 1.有没有用作者的合成代码 Sorry this code can not be applied on author's data. I did some modify and try on author's data, but the result is not good enough. And just like I mentioned in blog, composite in advance will cause ground truth shift, so I don't recommend you composite offline. Run-time composition is better。 2.谁的泛化性能更好 It's really hard to say. Maybe using different way of composition will cause some bias of neural network(because of bias in training data). In my case, this model generalize not bad on my data. So u can try both if time is enough.

[bhack]

@Joker316701882 Can we integrate a little bit more with https://github.com/hector-sab/DIM_DataCreation? Just to have a common reference cause the original dataset was unavailable. @hector-sab Have you tried to train it?

[Joker316701882]

@bhack The original dataset is available. You can apply it here https://sites.google.com/view/deepimagematting

[bhack]

On request.. have you trained on it?

[bitname]

@Joker316701882 Thank you. I will research it further.

[engrchrishenry]

@Joker316701882 Thank you for the code. Is it possible that you upload the modified code that you used on Author's data?

[michaelKurowski]

How to generate masks using pretrained model from README?

[Joker316701882]

@michaelKurowski Assume that you are going to run test.py Actually mask is not needed. But in this code, mask is used to generate trimap. You can see that code in test.py. In other cases, pretrained model is not necessary.

[ronyuzhang]

On testing step， why do you need input alpha image，i cant understand it.

[Joker316701882]

@ronyuzhang To generate trimap : )

[ronyuzhang]

this paper cant be called ‘auto image matting’，only if input the rgb image ，it is good

[Joker316701882]

@ronyuzhang 'Only input rgb to generate good mask' I think this problem hasn't been solved yet : )

[ronyuzhang]

"directly compute the alpha matte given an input image and trimap" ,why do you need input rgb and alpha matte to generate trimap?

[ronyuzhang]

@Joker316701882

[sanggehouzhihoujue]

@Joker316701882 the paper is to use the trimap and rgb to generate alpha matte，why do you aim to generate trimap？？？

[elzeru]

The purpose of the algorithm is to create the mask of an object present an image. The algorithm cannot guess what is the object you want to extract, so the trimap is here to give the algorithm an approximate position of it.

[lzh19961031]

@Joker316701882 @bitname Hi,我有个小问题。我现在想把一张图片中的物体加到另一张图片中的某个位置，请问我需要怎么做呢？我没有找到论文作者提供的代码？