Tiles are not correctly lined up with each other

[ProGamerGov]

Non feathered image:

Feathered image on-top of non-feathered image:

[ProGamerGov]

@0000sir I lack a lot of experience with ImageMagick, so I am unsure of what is causing this issue.

[ProGamerGov]

@0000sir Looking into the issue farther, could you help me understand what the values for w_percent and h_percent are? I think I have figured out the issue and solution, but I need to understand these two values to be sure.

w_percent=`echo 20 $w2 | awk '{print $1/$2}'`
h_percent=`echo 20 $h2 | awk '{print $1/$2}'`

The value for $w2 and $h2 comes from the height and width of tile 0.

[0000sir]

I saw this yesterday, but didn't have any minute to reply.. I think the problem is the "border", I didn't know really much about ImageMagick, but the I can answer the last question:

convert $out_file -crop 3x3+20+20@ +repage +adjoin $out_dir/$clean_name"_%d.jpg"

In this line, I split the first image ( which is 10001000 ) to a 3x3 array, with an expanded border on each block, that means each block is larger than 333333, the real size is 373*373.

then I run neural-style again, and generate 9 images of 1000*1000 from the last 3x3 array.

Then we have to combine these 9 images to a 3000*3000 image, we need to know the border width, it's not 20px anymore, so I calculated the w_percent and h_percent, that's how many the border (20px in my situation) takes up of each side.

but why we need borders, that's because we need larger size if we want to feather and combine the tiles.

Let me know if I didn't explain clearly

[ProGamerGov]

@0000sir So here is what I think the issue is. Looking at the 9 tiles which make up the above output image:

The 3x3 shape and and how the tile sizes fit into it:

677x1000 680x1000 677x1000
679x1000 682x1000 679x1000
677x1000 680x1000 677x1000

As per the naming use with $clean_name'_0.png, where $clean_name is the name of your input file, the corresponding number to each sized tile:

0: 677x1000 
1: 680x1000 
2: 677x1000
3: 679x1000 
4: 682x1000 
5: 679x1000
6: 677x1000 
7: 680x1000 
8: 677x1000

You can see that the size of each tile varies a bit. From looking at the code in the script, it looks like it is only using tile 0 for height and width values. You can see this starting with these lines of code:

w2=`convert $out_dir/$clean_name'_0.png' -format "%w" info:`
h2=`convert $out_dir/$clean_name'_0.png' -format "%h" info:`

Tiles 0,2,6,8 are: 667x1000 Tiles 1,7 are: 680x1000 Tiles 3,5 are: 679x1000 Tile 4 is: 682x1000

As you can see, the border values that the script calculates, only work with tiles 0,2,6,8 and not tiles 1,3,4,5,7. This is where the issue is, I think.

As per the planned features of choosing 2x2, 3x3, 4x4, etc... tiles, fixing this issue should keep in mind this planned feature.

---

Also, while I was reviewing the code, I see there are many things that could be better done when I paste it into: https://www.shellcheck.net (It's a spellchecker for bash code). It doesn't show any major issues, but it does suggest using $(code here) instead of 'code here'. Specifically see: https://github.com/koalaman/shellcheck/wiki/SC2086 and https://github.com/koalaman/shellcheck/wiki/SC2006

[ProGamerGov]

@0000sir Also, I submitted pull request with my changes thus far: https://github.com/0000sir/larger-neural-style/pull/9

So that if you try to help solve this tile issue, our two branches will not be incompatible.

---

The included features in the pull request are:

1. Overlapping the feathered and non feathered output images so that you can easily disguise feathering and not get "white ghostly/foggy lines" between tiles with different feathering values.
2. The Neural-Style command used on the tiles, and the original output's Neural-Style command, are now two separate commands. This opens up numerous possibilities like ensuring that the tiles do not diverge by using a lower iteration count than the iteration count used for the original output.
3. Numerous changes to the README.md file along with spelling and grammar fixes.
4. The script now automatically checks for the "output" directory and if it does not exist, the script creates it for you.
5. All code using the JPG format, has been changed so that it uses the PNG format. This is because the JPG format is more for sharing smaller lower quality images, while PNG format and other formats are made to store far more information, and thus are better suited for things like art creation.

[0000sir]

Great @ProGamerGov , I never thought the tiles were generated in different sizes, I don't know how did this happened... 😕

In your test result, the tiles neural-styled in step 3 are different in columns nor rows, are they same sized after step 2 ?

I'm going to find it out.

Thank you for correcting me in grammar and spelling, I'm not good at English language, shell craps too.. 🦀

I'm glad you're making this project better

[ProGamerGov]

@0000sir So I tried the script with a square content image:

1000x1000 997x1000 1000x1000
1000x997 1000x1000 1000x997
1000x1000 997x1000 1000x1000

Changing the width to the original height value, and the height to the original width value, results in the same pattern again.

1000x667 1000x679 1000x667
1000x680 1000x682 1000x680
1000x667 1000x679 1000x667

There appears to be something causing the tiles to have slightly different sizes. But the same pattern emerges.

Looking on the ImageMagick page for Montage, I wonder if it is possible to have different border values for each tile? Though if the pattern will always repeat itself, we only need 4 different border values. Though I am not sure that would work unless we know the cause of this strange pattern. All the tiles are the same size before they are run through Neural-Style, so it does not appear to be caused by ImageMagick.

I wonder if Neural-Style's code is causing this aspect ratio pattern, or the neural network models themselves are? Or maybe it's one of the other libraries Neural-Style uses, that causes it?

[ProGamerGov]

Looking at the original tiles for the square content image:

347x347 346x347 347x347 
347x346 346x346 347x346
347x347 346x347 347x347 

The tile sizes before putting them through Neural-Style, for the first image at the top of this post:

235x347 236x347 235x347
235x346 236x346 235x346 
235x347 236x347 235x347

The issue may actually be caused by ImageMagick.

[ProGamerGov]

Ok, so using this code: https://github.com/ProGamerGov/larger-neural-style/blob/master/bigbrush.sh#L47-L56 I have fixed the aspect ratio pattern. I believe it is caused by an ImageMagick bug, and thus I had to manually correct every tile to fix it. However, when the images are put back together, they are still not aligned properly.

Edit:

I also now think I see what the issue is. The values used for putting the image back together use the original tiles, and not the new ones.

[ProGamerGov]

@0000sir I fixed the issue! https://github.com/ProGamerGov/larger-neural-style/commit/b40ee02467f7b4b7beae3e0133e3f5a61722bde5

Though it can probably be done a lot more neater.

Part of the issue was that the script originally was using the original tile's length and width for putting the sites back together, instead of the new tile size.

[ProGamerGov]

I tried using an overlap value of 50, instead of 20, and I cannot see the lines between tiles using only 50 iterations for both the original output and the tiles:

Link to the image: http://i.imgur.com/W0cEKiZ.jpg

---

I now am thinking that it would be a good idea to make the overlap value a variable, so the script user can easily change it to whatever he/she wants. Rather than having to change the value on 4 different lines of code.

My fix at the moment is extremely messy, so it would probably be best if we could clean up the code a bit.

---

The pull request for the tiling alignment fix can be found here: https://github.com/0000sir/larger-neural-style/pull/10

[0000sir]

@ProGamerGov I can't believe you fix this so quickly! I was wondering if we change the image size from 1000 to 999, it should works better, as we're generating a 3x3 square.

I didn't have a chance to test it, may be I can help to clean the script after this weekend. My test environment is located in the office, and it was powered off... I can't connect to it 😭

[ProGamerGov]

@0000sir I considered changing the image size, but based on my past experiments with Neural-Style, I feel that allowing the user to have as much control as possible over variables like image size, is the best way to do it. The image sizes of the tiles were normally only a few single digit numbers off of the correct size due to what I suspect is an ImageMagick bug (Someone should report it to the devs), and thus my quick fix of resizing all the original tiles does the trick for fixing the issue.

[0000sir]

Yes, we can read image size and tiles amount from params, like --image-size 1000 and --tiles 3x3.

But it will be more difficult for people to understand how it works. Keep these in script, people run it and let the script do the magic.

If someone want to change the image size or something else, code is a better readme file.