Faithful Vision-Language Interpretation via Concept Bottleneck Models

Songning Lai, Lijie Hu, Junxiao Wang, Laure Berti and Di Wang

The Twelfth International Conference on Learning Representations (ICLR2024). (CCF None)

We introduce the Faithful Vision-Language Concept (FVLC) model, addressing the instability of label-free Concept Bottleneck Models (CBMs). Our FVLC model demonstrates superior stability against input and concept set perturbations across four benchmark datasets, with minimal accuracy degradation compared to standard CBMs, offering a reliable solution for model interpretation.

Towards Multi-dimensional Explanation Alignment for Medical Classification

Lijie Hu†, Songning Lai†, Wenshuo Chen†, Hongru Xiao, Hongbin Lin, Lu Yu, Jingfeng Zhang, and Di Wang

The Conference on Neural Information Processing Systems (NeurIPS 2024).(CCF A)

• We proposed an end-to-end framework called Med-MICN, which leverages the strength of different XAI methods such as concept-based models, neural symbolic methods, saliency maps, and concept semantics.
• Our outputs are interpreted in multiple dimensions, including concept prediction, saliency maps, and concept reasoning rules, making it easier for experts to identify and correct errors.
• Med-MICN demonstrates superior performance and interpretability compared with other concept-based models and the black-box model baselines.

DRIVE: Dependable Robust Interpretable Visionary Ensemble Framework in Autonomous Driving

Songning Lai†, Tianlang Xue, Hongru Xiao, Lijie Hu, Jiemin Wu, Ruiqiang Xiao, Ninghui Feng, Haicheng Liao, Zhenning Yang, Yutao Yue~

The Conference on (ICRA 2025).(CCF B)

We introduce DRIVE, a framework designed to enhance the dependability and stability of explanations in end-to-end unsupervised autonomous driving models, addressing instability issues and improving trustworthiness through consistent and stable interpretability and output, as demonstrated by empirical evaluations. This framework provides novel metrics for assessing the reliability of concept-based explainable autonomous driving systems, advancing their real-world deployment.

PEPL: Precision-Enhanced Pseudo-Labeling for Fine-Grained Image Classification in Semi-Supervised Learning

Bowen Tian†, Songning Lai†, Lujundong Li, Zhihao Shuai, Runwei Guan, Tian Wu, Yutao Yue~

The Conference on (ICASSP 2025).(CCF B)

We introduce Precision-Enhanced Pseudo-Labeling (PEPL), a semi-supervised learning approach for fine-grained image classification that generates and refines pseudo-labels using Class Activation Maps (CAMs) to capture essential details, significantly improving accuracy and robustness over existing methods on benchmark datasets. The approach consists of initial and semantic-mixed pseudo-label generation phases to enhance the quality of labels and has been open-sourced for public use.

Predicting Lysine Phosphoglycerylation Sites using Bidirectional Encoder Representations with Transformers & Protein Feature Extraction and Selection

Songning Lai, Xifeng Hu, Jing Han, Chun Wang, Subhas Mukhopadhyay, Zhi Liu~ and Lan Ye~

2022 15th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI 2022).(Tsinghua B)

We developed a model using protein feature acquisition, F_Score selection, KNN cleaning, and SMOTE for positive sample synthesis, combined with BERT classification based on Transformer, achieving an accuracy of up to 99.61% and an MCC of 99.1%, surpassing previous models. This work demonstrates significant potential for future applications.

Shared and private information learning in multimodal sentiment analysis with deep modal alignment and self-supervised multi-task learning

Songning La†, Jiakang Li, Guinan Guo, Xifeng Hu, Yulong Li, Yuan Tan, Zichen Song, Yutong Liu, Zhaoxia Ren~, Chun Wang~, Danmin Miao~ and Zhi Liu~

International Joint Conference on Neural Networks (IJCNN 2024). (CCF C)

We propose a deep learning module that captures shared information across modalities using a covariance matrix and introduces a self-supervised label generation module to extract modality-specific private information, enhancing multimodal sentiment analysis performance through multi-task learning. Extensive experiments on benchmark datasets demonstrate the model’s effectiveness in capturing subtle multimodal sentiments.

A Comprehensive Review of Community Detection in Graphs

Jiakang Li†, Songning Lai†, Zhihao Shuai, Yuan Tan, Yifan Jia, Mianyang Yu, Zichen Song, Xiaokang Peng, Ziyang Xu, Yongxin Ni, Haifeng Qiu, Jiayu Yang, Yutong Liu, Yonggang Lu~

Neurocomputing (JCR Q1 (IF: 6.0) CCF C)

This review explores community detection in graphs, covering methods such as modularity-based, spectral clustering, probabilistic modelling, and deep learning, and introduces a new method, while comparing performances across datasets with and without ground truth. The review offers a comprehensive understanding of the current landscape in community detection techniques.

Multimodal Sentiment Analysis: A Survey

Songning Lai, Haoxuan Xu, Xifeng Hu, Zhaoxia Ren~ and Zhi Liu~

Displays (JCR Q1 (IF: 4.3))

This review provides an overview of multimodal sentiment analysis, covering its definition, history, recent datasets, advanced models, challenges, and future prospects, offering guidance on promising research directions. The review aims to support researchers in developing more effective models in this rapidly evolving field.

Cross-domain car detection model with integrated convolutional block attention mechanism

Haoxuan Xu†, Songning Lai† and Yang Yang~

Image and Vision Computing (JCR Q1 (IF:4.7) CCF C)

We propose a Cross-Domain Car Detection Model with an integrated convolutional block Attention mechanism (CDCDMA), featuring a complete cross-domain detection framework, unpaired target domain image generation emphasizing car headlights, GIOU loss function, and a two-headed CBAM, which improved detection performance by 40% over non-CDCDMA models on the SODA 10 M and BDD100K datasets. This model significantly enhances cross-domain car recognition, outperforming most existing advanced models.

Enhancing domain adaptation for plant diseases detection through Masked Image Consistency in Multi-Granularity Alignment

Guinan Guo, Songning Lai, Qingyang Wu,, Yuntao Shou, and Wenxu Shi

Expert Systems With Applications (JCR Q1 (IF:8.4) CCF C)

This study proposes MIC-MGA, a deep learning approach combining Multi-Granularity Alignment and Masked Image Consistency, to enhance cross-domain generalization in plant disease detection. The method improves object detector architecture while integrating domain adaptation and style-augmented training to address feature distribution shifts across different environmental conditions. Experimental results demonstrate state-of-the-art performance in cross-domain detection tasks (highest mAP), showing significant potential for improving disease prevention and sustainable agriculture.

CAT: Concept-level backdoor ATtacks for Concept Bottleneck Models

Songning Lai, Jiayu Yang, Yu Huang, Lijie Hu, Tianlang Xue, Zhangyi Hu, Jiaxu Li, Haicheng Liao, Yutao Yue~

We introduce CAT and CAT+, methodologies for concept-level backdoor attacks on Concept Bottleneck Models (CBMs), which use conceptual representations to embed triggers during training, allowing for controlled manipulation of predictions; our evaluation shows high attack success rates and stealthiness, highlighting potential security risks in CBMs.

Wolf2Pack: The AutoFusion Framework for Dynamic Parameter Fusion

Bowen Tian†, Songning Lai†, Yutao Yue

We introduce AutoFusion, a framework that dynamically permutes parameters across layers of models with the same architecture, enabling multi-task learning without pre-training, and showing superior performance on benchmark datasets compared to methods like Weight Interpolation and Git Re-Basin. This framework provides a scalable solution for integrating models, suitable for advancing research and practical use.

TimeSieve: Extracting Temporal Dynamics through Information Bottleneck

Ninghui Feng†, Songning Lai†, Fobao Zhou, Zhenxiao Yin, Hang Zhao~

This paper introduces TimeSieve, an innovative time series forecasting model that uses wavelet transforms and information bottleneck theory to capture multi-scale features and filter redundant information, significantly improving accuracy and generalization without manual hyperparameter tuning, thus outperforming existing methods in most cases.

FTS: A FRAMEWORK TO FIND A FAITHFUL TIMESIEVE

Songning Lai, Ninghui Feng, Jiechao Gao, Hao Wang, Haochen Sui, Xin Zou, Jiayu Yang, Wenshuo Chen, Hang Zhao, Xuming Hu, Yutao Yue~

• Our research provides faithful technical support and theoretical support to the field of time series forecasting, promising to advance the development and reliability of forecasting methods within the industry. Through these efforts, we aim to bolster the trustworthiness of models, ultimately supporting decision-making processes that rely on accurate and consistent predictions.

Maintaining Informative Coherence: Migrating Hallucinations in Large Language Models via Absorbing Markov Chains

Jiemin Wu, Songning Lai, Ruiqiang Xiao, Tianlang Xue, Jiayu Yang, Yutao Yue

We propose a novel decoding strategy using absorbing Markov chains to measure and enhance the fidelity of contextual information in Large Language Models (LLMs), reducing hallucinations without additional training, and demonstrating superior performance in maintaining accuracy on benchmarks like TruthfulQA and FACTOR.

RMDM: Radio Map Diffusion Model with Physics Informed

Haozhe Jia, Wenshuo Chen, Zhihui Huang, Hongru Xiao, Nanqian Jia, Keming Wu, Songning Lai, Yutao Yue

Radio map reconstruction is essen- tial for enabling advanced applications, yet chal- lenges such as complex signal propagation and sparse data hinder accurate reconstruction. To ad- dress these issues, we propose the Radio Map Diffusion Model (RMDM), a physics-informed framework that integrates Physics-Informed Neu- ral Networks (PINNs) to incorporate constraints like the Helmholtz equation. RMDM employs a dual U-Net architecture: the first ensures physical consistency by minimizing PDE residuals, bound- ary conditions, and source constraints, while the second refines predictions via diffusion-based de- noising. By leveraging physical laws, RMDM sig- nificantly enhances accuracy, robustness, and gen- eralization.

Free-T2M: Frequency Enhanced Text-to-Motion Diffusion Model With Consistency Loss

Wenshuo Chen, Haozhe Jia, Songning Lai, Keming Wu, Hongru Xiao, Lijie Hu, Yutao Yue

Rapid progress in text-to-motion generation has been largely driven by diffusion models. However, existing methods focus solely on temporal modeling, thereby overlooking frequency-domain analysis. We identify two key phases in motion denoising: the semantic planning stage and the fine-grained improving stage. To address these phases effectively, we propose Frequency enhanced text-to-motion diffusion model (Free-T2M), incorporating stage-specific consistency losses that enhance the robustness of static features and improve fine-grained accuracy.

SegACIL: Solving the Stability-Plasticity Dilemma in Class-Incremental Semantic Segmentation

Jiaxu Li†, Songning Lai†, Rui Li, Di Fang, Kejia Fan, Jianheng Tang, Yuhan Zhao, Rongchang Zhao, Dongzhan Zhou, Yutao Yue, Huiping Zhuang

We propose SegACIL, a novel continual learning method for semantic segmentation using a linear closed-form solution, requiring only a single epoch and reducing computational costs. SegACIL achieves performance on par with joint learning, effectively balancing stability and plasticity by retaining prior knowledge while adapting to new data, addressing the stability-plasticity dilemma.

Learning New Concepts, Remembering the Old: A Novel Continual Learning

Songning Lai, Mingqian Liao, Zhangyi Hu, Jiayu Yang, Wenshuo Chen, Yutao Yue

To enhance Concept Bottleneck Models (CBMs) for evolving data streams, we propose CONceptual Continual Incremental Learning (CONCIL), a framework that accumulates new concepts and classes while preventing catastrophic forgetting. CONCIL reformulates updates as linear regression problems, eliminating gradient-based updates and requiring only recursive matrix operations, ensuring computational efficiency. Suitable for real-time and large-scale applications, CONCIL achieves “absolute knowledge memory” and outperforms traditional CBMs in concept- and class-incremental settings, setting new benchmarks for continual learning.

Beyond Task Vectors: Selective Task Arithmetic Based on Importance Metrics

Tian Bowen†, Lai Songning†, Wu Jiemin, Shuai Zhihao, Ge Shiming, Yue Yutao

This paper introduces Selective Task Arithmetic (STA), a training-free framework enhancing multi-task performance via task-specific parameter fusion. STA addresses parameter importance diversity using a loss-sensitive metric from a first-order Taylor expansion, reduces over-reliance on hyperparameter tuning by improving task vector sparsity, and mitigates task forgetting more effectively by leveraging parameter importance to minimize noisy elements, improving model generalization and robustness.

Guarding the Gate: ConceptGuard Battles Concept-Level Backdoors in Concept Bottleneck Models

Songning Lai, Yu Huang, Jiayu Yang, Gaoxiang Huang, Wenshuo Chen, Yutao Yue

Our contributions are threefold: (i) we present ConceptGuard as the first defense mechanism tailored for concept-level backdoor attacks in CBMs; (ii) we provide theoretical guarantees that ConceptGuard can effectively defend against such attacks within a certain trigger size threshold, ensuring robustness; and (iii) we demonstrate that ConceptGuard maintains the high performance and interpretability of CBMs, crucial for trustworthiness.

DRIVE: Dual-Robustness via Information Variability and Entropic Consistency in Source-Free Unsupervised Domain Adaptation

Ruiqiang Xiao†, Songning Lai†, Yijun Yang, Jiemin Wu, Yutao Yue, Lei Zhu

To address challenges in Source-Free Unsupervised Domain Adaptation (SFUDA), we propose DRIVE, a dual-model framework leveraging mutual information and entropy. DRIVE uses two identically initialized models; one is perturbed via PGD guided by mutual information to focus on high-uncertainty regions, while an entropy-aware pseudo-labeling strategy adjusts label weights based on prediction uncertainty. DRIVE’s two-stage adaptation process first aligns models on stable features using mutual information consistency loss, then dynamically adjusts perturbation levels to enhance generalization and robustness. Evaluations show DRIVE outperforms existing methods in adaptation accuracy and stability across complex domains.

TS-ACL: A Time Series Analytic Continual Learning Framework for Privacy-Preserving and Class-Incremental Pattern Recognition

Kejia Fan†, Jiaxu Li†, Songning Lai†, Linpu Lv, Anfeng Liu, Jianheng Tang, Houbing Herbert Song, Huiping Zhuang

To tackle catastrophic forgetting in class-incremental time series pattern recognition, we propose TS-ACL, a privacy-preserving, gradient-free continual learning framework. TS-ACL uses a pre-trained frozen encoder for embedding extraction and updates an analytic classifier recursively with a closed-form solution, avoiding catastrophic forgetting without storing historical data. This lightweight approach ensures non-forgetting, privacy preservation, and efficiency, suitable for edge computing. Experiments on five benchmarks demonstrate superior performance compared to existing methods.