A Multi-Feature Image Captioning Model with Common-Specific Collaborative Modeling and Mixture-of-Experts Fusion

Existing image captioning methods often struggle to simultaneously capture global semantics and fine-grained details, leading to overly generic descriptions or missing key information. To address this issue, we propose EAFCC, short for Expert-Aware Fusion with Common-Specific Collaboration, a multi-feature image captioning model with mixture-of-experts fusion. The model takes grid features, region features, and segmentation features as parallel inputs. In the encoding stage, a pre-fusion module first performs implicit alignment of heterogeneous visual features and models cross-feature common information within a shared semantic space, while preserving the basis for feature-specific representations. A shared Transformer encoder is then employed to learn the common representations, whose consistency and discriminability are further enhanced by a contrastive learning constraint. Meanwhile, three independent encoding branches are used to model feature-specific representations, so as to retain fine-grained complementary information associated with different feature types. A gated routing mechanism inspired by mixture-of-experts is further introduced to dynamically weight and fuse the specific information. Finally, the enhanced common features and the fused specific features are collaboratively integrated, processed by a skip-connected encoder, and fed into a Transformer decoder to generate image captions. Experimental results on the MS-COCO dataset show that EAFCC achieves competitive performance on the CIDEr-D metric and outperforms existing single-feature and dual-feature methods, demonstrating the effectiveness of common-specific collaborative modeling and dynamic multi-feature fusion for image captioning.