跨模态生成

主要围绕文本-图像的跨模态生成开展研究，涵盖了动漫、风景、食物生成等多种类别，以及修复、上色、超分、美化、三维生成和序列图像生成等多种类型。提出了以细粒度解耦为代表的一系列方法，有效解决了内容语义不准、细节重复生成等难题，在保证生成准确率的前提下显著提升了图像生成效率。如图所示，本方向形成了从底层架构到基线模型再到应用拓展的系统性研究框架，并支撑了个性化图像生成平台“即绘”。成果在国内外学术界产生广泛影响。

个性化图像生成平台“即绘”：【平台介绍】【校内访问试用】【校外访问试用】

代表性论文

[1] Ming Tao, Bing-Kun Bao*, Hao Tang, Yaowei Wang, Changsheng Xu. CoIn: A Lightweight and Effective Framework for Story Visualization and Continuation. ACM International Conference on Multimedia (ACM MM) 2024【论文】【Github】

[2] Ming Tao, Bing-Kun Bao*, Hao Tang, Yaowei Wang, Changsheng Xu. StoryImager: A Unified and Efficient Framework for Coherent Story Visualization and Completion. European Conference on Computer Vision (ECCV) 2024【论文】【Github】 

[3] Mengling Xu, Jie Wang, Ming Tao, Bing-Kun Bao*, Changsheng Xu. CookGALIP: Recipe Controllable Generative Adversarial CLIPs with Sequential Ingredient Prompts for Food Image Generation. IEEE Transactions on Multimedia (TMM) 2024【论文】【Github】

[4] Yefei Sheng, Ming Tao, Jie Wang, Bing-Kun Bao*. ISF-GAN: Imagine, Select, and Fuse with GPT Based Text Enrichment for Text-to-Image Synthesis. ACM Transactions on Multimedia Computing, Communications and Applications (ACM TOMM) 2024 【论文】【Github】

[5] Ming Tao, Bing-Kun Bao*, Hao Tang, Changsheng Xu. GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesis. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023 【论文】【Github】

[6] Ming Tao, Bing-Kun Bao*, Hao Tang, Fei Wu, Longhui Wei, Qi Tian: DE-Net: Dynamic Text-guided Image Editing Adversarial Networks. AAAI Conference on Artificial Intelligence (AAAI) 2023 【论文】【Github】

[7] Bowen Yuan, Yefei Sheng, Bing-Kun Bao*, Yi-Ping Phoebe Chen, Changsheng Xu. Semantic Distance Adversarial Learning for Text-to-Image Synthesis. IEEE Transactions on Multimedia (TMM) 2023 【论文】【Github】

[8] Ming Tao, Hao Tang, Fei Wu, Xiaoyuan Jing, Bing-Kun Bao*, Changsheng Xu. DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022 (Oral) 【论文】【Github】