• 中国精品科技期刊
  • CCF推荐A类中文期刊
  • 计算领域高质量科技期刊T1类
Advanced Search
Zhang Hongbin, Ji Donghong, Yin Lan, Ren Yafeng, Niu Zhengyu. Caption Generation from Product Image Based on Tag Refinement and Syntactic Tree[J]. Journal of Computer Research and Development, 2016, 53(11): 2542-2555. DOI: 10.7544/issn1000-1239.2016.20150906
Citation: Zhang Hongbin, Ji Donghong, Yin Lan, Ren Yafeng, Niu Zhengyu. Caption Generation from Product Image Based on Tag Refinement and Syntactic Tree[J]. Journal of Computer Research and Development, 2016, 53(11): 2542-2555. DOI: 10.7544/issn1000-1239.2016.20150906

Caption Generation from Product Image Based on Tag Refinement and Syntactic Tree

More Information
  • Published Date: October 31, 2016
  • Automatic caption generation from product image is an interesting and challenging research task of image annotation. However, noisy words interference and inaccurate syntactic structures are the key problems that affect the research heavily. For the first problem, a novel idea of tag refinement (TR) is presented: absolute rank (AR) feature is applied to strengthen the key words weights. The process is called the first tag refinement. The semantic correlation score of each word is calculated in turn and the words that have the tightest semantic correlations with images content are summarized for caption generation. The process is called the second tag refinement. A novel natural language generation (NLG) algorithm named word sequence blocks building (WSBB) is designed accordingly to generate N gram word sequences. For the second problem, a novel idea of syntactic tree (ST) is presented: a complete syntactic tree is constructed recursively based on the N gram word sequences and predefined syntactic subtrees. Finally, sentence is generated by traversing all leaf nodes of the syntactic tree. Experimental results show both the tag refinement and the syntactic tree help to improve the annotation performance. More importantly, not only the semantic information compatibility but also the syntactic mode compatibility of the generated sentence is better retained simultaneously. Moreover, the sentence contains abundant semantic information as well as coherent syntactic structure.
  • Related Articles

    [1]Bi Fenglin, Zhang Qiming, Zhang Jiarui, Wang Yantong, Chen Yang, Zhang Yanbin, Wang Wei, Zhou Xuan. A Retrieval-Augmented Generation System Based on a Sliding Window Strategy in Large Language Models[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440411
    [2]Lin Meng, Dai Chengwei, Guo Tao. A Method for Generating Explanations of Offensive Memes Based on Multimodal Large Language Models[J]. Journal of Computer Research and Development, 2024, 61(5): 1206-1217. DOI: 10.7544/issn1000-1239.202330960
    [3]Chen Xuanting, Ye Junjie, Zu Can, Xu Nuo, Gui Tao, Zhang Qi. Robustness of GPT Large Language Models on Natural Language Processing Tasks[J]. Journal of Computer Research and Development, 2024, 61(5): 1128-1142. DOI: 10.7544/issn1000-1239.202330801
    [4]Yang Yi, Li Ying, Chen Kai. Vulnerability Detection Methods Based on Natural Language Processing[J]. Journal of Computer Research and Development, 2022, 59(12): 2649-2666. DOI: 10.7544/issn1000-1239.20210627
    [5]Pan Xuan, Xu Sihan, Cai Xiangrui, Wen Yanlong, Yuan Xiaojie. Survey on Deep Learning Based Natural Language Interface to Database[J]. Journal of Computer Research and Development, 2021, 58(9): 1925-1950. DOI: 10.7544/issn1000-1239.2021.20200209
    [6]Zheng Haibin, Chen Jinyin, Zhang Yan, Zhang Xuhong, Ge Chunpeng, Liu Zhe, Ouyang Yike, Ji Shouling. Survey of Adversarial Attack, Defense and Robustness Analysis for Natural Language Processing[J]. Journal of Computer Research and Development, 2021, 58(8): 1727-1750. DOI: 10.7544/issn1000-1239.2021.20210304
    [7]Bao Yang, Yang Zhibin, Yang Yongqiang, Xie Jian, Zhou Yong, Yue Tao, Huang Zhiqiu, Guo Peng. An Automated Approach to Generate SysML Models from Restricted Natural Language Requirements in Chinese[J]. Journal of Computer Research and Development, 2021, 58(4): 706-730. DOI: 10.7544/issn1000-1239.2021.20200757
    [8]Che Haiyan, Feng Tie, Zhang Jiachen, Chen Wei, and Li Dali. Automatic Knowledge Extraction from Chinese Natural Language Documents[J]. Journal of Computer Research and Development, 2013, 50(4): 834-842.
    [9]Shu Ting, Liu Lianggui, Xu Weiqiang, and Li Wenshu. Adaptive Executable Test Sequences Generation from an Extended Finite State Machine[J]. Journal of Computer Research and Development, 2012, 49(6): 1211-1219.
    [10]Lü Yanli, Li Xiaojian, Xia Chunhe, and Liu Shuzhi. Research on the Security of Initial Sequence Number Generation Arithmetic[J]. Journal of Computer Research and Development, 2005, 42(11): 1940-1945.

Catalog

    Article views (1381) PDF downloads (476) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return