Idea of automatic coloring tool for illustration

[hepesu]

Automatic coloring tool can be disassembled into several parts, line-closing tool,floodfill tool, area classification tool, color design tool, shading tool. There are a lot of tools to fill the area, such as MultiFill, but these tools need closing line-art. The deep learning tool can now complete the parts besides shading. As for tools in project HAT, LineRelifer and LineFiller can be used to complete the basic color filling, which fulfill line-closing and floodfill parts. ​​​​

As for full version of LineFiller, it will contain a model performs area classification. And JACS provides a model for color design, as well as unpublished LineShader to complete the shading. There are currently two implementations of LineFiller, one is traditional algorithm for filling and a convolution neural network model for classification, the other one is based on reinforcement learning.

Back to the end-to-end model based on deep learning, in fact if you have tried, your might find that the color design of model is weak, leads to lots of instructions for color suggestion. And sometimes due to incomplete classification of area, more instructions needed. So the total efficiency might be not as good as the paint bucket. If you want to try new color palette, you would have to add more suggestions. Therefore, in actual pipeline, automatic coloring workflow needs to be break into parts, which can be guided by human in any step and if a problem is solved, it will not appear in next step.

As to the coloring style which deep learning models have, full version of LineFiller only to finish the Cel style. However, it is possible to get styled color result without deep learning (see the Blur filter of CelFX). And various style models can be trained, CAN is a potential structure for this task.

To sum up, in this way, every steps are available for human guide, and after finishing the Cel shade, you can get any style with transformation model. It is suitable for production pipeline.