dcscn-super-resolution
dcscn-super-resolution copied to clipboard
jiny2001
Possible to export to Android
Is it somehow possible to export this model for Android use? I guess it might be doable with Tensorflow Mobile or Tensorflow Lite. Then it would be needed to get hold of checkpoints files (.cpkt) as well as a frozen graph?
Hi, I'm also curious about porting to mobile. A year ago I have tried TensorFlow RT with freezer graph. But I believe Tensorflow Lite is the we should use? In this case do we just need to convert the checkpoint file??
Yes, Tensorflow Lite is the new framework for this. Both checkpoint files (.cpkt) and a graph def (.pb) is needed to get a freezed graph. This freezed graph can be used (and also optimized) to a Tensorflow Lite file.
So, do you mean you want to add freezed graph output feature on this project? Or you just want to confirm if it just runs directory under tensorflow lite?
Well, I was wondering how you would manage to get a frozen graph from the existing model. The checkpoint files should not be a problem but the graph def (.pb) is not as straight forward.
You can see how the structure for converting looks here (visualized): https://www.tensorflow.org/lite/convert
Well,,, when I tried freeze_graph function on the other project, I just used this freeze_graph.py. And it worked for TF RT. I succeed to make it run on Mac witch C++ in realtime. Sorry for not sure about TF Lite.
But I believe you can also check freeze_graph_test.py so that you can export it as freezed graph.
Okey, thanks! But for example when trying to use the function you provided you need to know output_node_names and in other similar functions also input_nodes. How do you get hold of those? In Tensorboard the graph got a node called output for instance which contains 5 different outputs. Do you know what do provide as output_node_name?
Y_/output/Mean ?×?×?×1
Y_/output/sub ?×?×?×1
diff ?×?×?×1
gradients/diff_grad/Shape ?×?×?×1
Y_/output/dcscn_L8_F96to48_NIN_A64_PS_R1F32
Hi, sorry for late reply. I'm so busy these days... Yes you need the name of the last output node. I think what you want to use is this node. https://github.com/jiny2001/dcscn-super-resolution/blob/master/DCSCN.py#L265
In this case, the name is just "output". Please try it.
I ported to pure keras https://github.com/iperov/DeepFaceLab/blob/master/imagelib/DCSCN.py real model size is 6mb without optimizer weights works on plaidML backend
Wow!! hi iperov, this is AWSOME!!
I guess you can see the super resolution result better when you use an image which has more clear edges (the sample face image has blurred so it's hard to see the difference). However, the project is also awesome. I pushed a star on your project and will add to my readme that you can use my model via your project. Thank you so much!!
@jiny2001 is DCSCN better than for example SRGAN ? did you compare? do you know any superresolution network that will work better for blurred images? do you know any superresolution network that can add details/texture to blurred images?
Hi, if you compare DCSCN and SRResNet, SRResNet will have a better result. However, it might be difficult to see the difference from human eyes. And also normal SR network won't work better on blurred image.
SRGAN will normally have less PSNR than other SR networks, however, they generate details/texture and makes it more "natural" so it might work on blurred image. But I haven't tested. For the face super resolution, I believe those papers are the best for now. But it only works in certain angle and resolution I guess. http://openaccess.thecvf.com/content_cvpr_2018/CameraReady/1035.pdf https://arxiv.org/pdf/1711.10703.pdf
Anyway, I guess you can try SRGAN since there are some implementations in github. Thx.
my opinion is that all SRs are actually trained to restorate downscaled images, so these images have high density of "details per pixel". But real world images are not downscaled images :D
especially bicubic downsampled image has more information about upscaled image than bilinear or nearest-neighbor. So why to train SR to restorate images that already have more information than real world image? My suggestion to train x2 DCSCN upscale network on x4 nearest neighbor downscaled (random offset 1-4px), then x2 bilinear upscaled images
You are right. I agree I should train from bilinear. I'm just using bicubic because every other people use it and I can compare the performance with it. let me know if you can point me a nice training data set. I can try it with bilinear downsampling. ;)
Okey, thanks! But for example when trying to use the function you provided you need to know output_node_names and in other similar functions also input_nodes. How do you get hold of those? In Tensorboard the graph got a node called output for instance which contains 5 different outputs. Do you know what do provide as output_node_name?
Y_/output/Mean ?×?×?×1Y_/output/sub ?×?×?×1diff ?×?×?×1gradients/diff_grad/Shape ?×?×?×1Y_/output/dcscn_L8_F96to48_NIN_A64_PS_R1F32
have you found the input node name?
Hi @ricardobnjunior, sorry for late reply. The input node is the one which is defined as placeholder, the name is "x". I guess you have found this already though...
Hi @iperov, the result is really awesome!! Great!! I guess much much smaller model will also work if it focused on face super resolution. Which will greatly improve processing speed. Will try it when i have a time. Thx!!
Hi @ricardobnjunior, sorry for late reply. The input node is the one which is defined as placeholder, the name is "x". I guess you have found this already though...
Hi @iperov, the result is really awesome!! Great!! I guess much much smaller model will also work if it focused on face super resolution. Which will greatly improve processing speed. Will try it when i have a time. Thx!!
Yes, I found! Thank you for reply ;)
my opinion is that all SRs are actually trained to restorate downscaled images, so these images have high density of "details per pixel". But real world images are not downscaled images :D
especially bicubic downsampled image has more information about upscaled image than bilinear or nearest-neighbor. So why to train SR to restorate images that already have more information than real world image? My suggestion to train x2 DCSCN upscale network on x4 nearest neighbor downscaled (random offset 1-4px), then x2 bilinear upscaled images
@iperov Could you please explain further about the rationale behind suggesting this - "x4 nearest neighbor downscaled (random offset 1-4px), then x2 bilinear upscaled images"
@Nik-Khandelwal
main idea is remove more information from downscaled image about it upscaled version, therefore neural network will not learn to fetch that info