A Coloring Algorithm for Flower Line Drawings with Meticulous Effect Based on Semantic Matching of Reference Images

The problem of coloring flower line drawings with meticulous effect based on a reference image is addressed. Existing reference-based coloring algorithms for line drawing are difficult to learn and simulate the unique color gradient effect of meticulous flower paintings. Moreover, the reference image in these algorithms is usually required to have similar geometric layout structure to the line drawing, which limits the applicability of the algorithms. Therefore, it is difficult to directly apply existing algorithms to accomplish coloring of line drawings with meticulous effect. On the basis of conditional generative adversarial network(CGAN) framework, a coloring algorithm for flower line drawings with meticulous effect is proposed by means of semantic matching between the reference image and the line drawing. In terms of network structure design, the proposed algorithm uses U-Net as the basis of the generator and designs two additional sub-modules. One is the semantic positioning sub-module. This module pre-trains a semantic segmentation network to generate a semantic label map of the flower line drawing. The label map is encoded as an adaptive instance normalization affine parameter and then introduced into the coloring model to improve the recognition ability of different semantic regions and the accuracy of color positioning. The other is the color coding sub-module. This module extracts the color features of the reference image, and then splices to the first three decoding layers of the generator, in which way, the color information is injected into the color model. Combining this module with semantic location module, our algorithm enhances the learning and simulation of gradient color pattern. In network training stage, the algorithm does not train the model on “original meticulous flower work-flower line drawing” data pairs. Instead, a perturbed version of the original work via such perturbation operations as disturbing the original geometric structure is generated and then “perturbed version-flower line drawing” data pairs are used to train our model, which turns out to reduce the model’s dependence on the spatial geometry layout of the original work and to then improve the applicability of the proposed algorithm. The experimental results show that the proposed algorithm has a correct response to the color semantics of the reference image selected by the user. It is also shown that the introduced structure of semantic positioning module and color coding module could improve the simulation effect of gradient colors and realize the colorization of the flower line drawing under the guidance of different reference images, as well as diversified coloring results.