Faithful Vision-Language Interpretation via Concept Bottleneck Models

Songning Lai†, Lijie Hu†, Junxiao Wang, Laure Berti and Di Wang

The Twelfth International Conference on Learning Representations (ICLR2024). (CCF None)

We introduce the Faithful Vision-Language Concept (FVLC) model, addressing the instability of label-free Concept Bottleneck Models (CBMs). Our FVLC model demonstrates superior stability against input and concept set perturbations across four benchmark datasets, with minimal accuracy degradation compared to standard CBMs, offering a reliable solution for model interpretation.

Towards Multi-dimensional Explanation Alignment for Medical Classification

Lijie Hu†, Songning Lai†, Wenshuo Chen†, Hongru Xiao, Hongbin Lin, Lu Yu, Jingfeng Zhang, and Di Wang

The Conference on Neural Information Processing Systems (NeurIPS 2024).(CCF A)

• We proposed an end-to-end framework called Med-MICN, which leverages the strength of different XAI methods such as concept-based models, neural symbolic methods, saliency maps, and concept semantics.
• Our outputs are interpreted in multiple dimensions, including concept prediction, saliency maps, and concept reasoning rules, making it easier for experts to identify and correct errors.
• Med-MICN demonstrates superior performance and interpretability compared with other concept-based models and the black-box model baselines.

Predicting Lysine Phosphoglycerylation Sites using Bidirectional Encoder Representations with Transformers & Protein Feature Extraction and Selection

Songning Lai, Xifeng Hu, Jing Han, Chun Wang, Subhas Mukhopadhyay, Zhi Liu~ and Lan Ye~

2022 15th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI 2022).(Tsinghua B)

We developed a model using protein feature acquisition, F_Score selection, KNN cleaning, and SMOTE for positive sample synthesis, combined with BERT classification based on Transformer, achieving an accuracy of up to 99.61% and an MCC of 99.1%, surpassing previous models. This work demonstrates significant potential for future applications.

Shared and private information learning in multimodal sentiment analysis with deep modal alignment and self-supervised multi-task learning

Songning Lai†, Jiakang Li†, Guinan Guo, Xifeng Hu, Yulong Li, Yuan Tan, Zichen Song, Yutong Liu, Zhaoxia Ren~, Chun Wang~, Danmin Miao~ and Zhi Liu~

International Joint Conference on Neural Networks (IJCNN 2024). (CCF C)

We propose a deep learning module that captures shared information across modalities using a covariance matrix and introduces a self-supervised label generation module to extract modality-specific private information, enhancing multimodal sentiment analysis performance through multi-task learning. Extensive experiments on benchmark datasets demonstrate the model’s effectiveness in capturing subtle multimodal sentiments.

A Comprehensive Review of Community Detection in Graphs

Jiakang Li†, Songning Lai†, Zhihao Shuai, Yuan Tan, Yifan Jia, Mianyang Yu, Zichen Song, Xiaokang Peng, Ziyang Xu, Yongxin Ni, Haifeng Qiu, Jiayu Yang, Yutong Liu, Yonggang Lu~

Neurocomputing (JCR Q1 (IF: 6.0) CCF C)

This review explores community detection in graphs, covering methods such as modularity-based, spectral clustering, probabilistic modelling, and deep learning, and introduces a new method, while comparing performances across datasets with and without ground truth. The review offers a comprehensive understanding of the current landscape in community detection techniques.

Multimodal Sentiment Analysis: A Survey

Songning Lai, Haoxuan Xu, Xifeng Hu, Zhaoxia Ren~ and Zhi Liu~

Displays (JCR Q1 (IF: 4.3))

This review provides an overview of multimodal sentiment analysis, covering its definition, history, recent datasets, advanced models, challenges, and future prospects, offering guidance on promising research directions. The review aims to support researchers in developing more effective models in this rapidly evolving field.

Cross-domain car detection model with integrated convolutional block attention mechanism

Haoxuan Xu†, Songning Lai† and Yang Yang~

Image and Vision Computing (JCR Q1 (IF:4.7) CCF C)

We propose a Cross-Domain Car Detection Model with an integrated convolutional block Attention mechanism (CDCDMA), featuring a complete cross-domain detection framework, unpaired target domain image generation emphasizing car headlights, GIOU loss function, and a two-headed CBAM, which improved detection performance by 40% over non-CDCDMA models on the SODA 10 M and BDD100K datasets. This model significantly enhances cross-domain car recognition, outperforming most existing advanced models.

CAT: Concept-level backdoor ATtacks for Concept Bottleneck Models

Songning Lai†, Jiayu Yang†, Yu Huang†, Lijie Hu, Tianlang Xue, Zhangyi Hu, Jiaxu Li, Haicheng Liao, Yutao Yue~

We introduce CAT and CAT+, methodologies for concept-level backdoor attacks on Concept Bottleneck Models (CBMs), which use conceptual representations to embed triggers during training, allowing for controlled manipulation of predictions; our evaluation shows high attack success rates and stealthiness, highlighting potential security risks in CBMs.

Wolf2Pack: The AutoFusion Framework for Dynamic Parameter Fusion

Bowen Tian†, Songning Lai†, Yutao Yue

We introduce AutoFusion, a framework that dynamically permutes parameters across layers of models with the same architecture, enabling multi-task learning without pre-training, and showing superior performance on benchmark datasets compared to methods like Weight Interpolation and Git Re-Basin. This framework provides a scalable solution for integrating models, suitable for advancing research and practical use.

PEPL: Precision-Enhanced Pseudo-Labeling for Fine-Grained Image Classification in Semi-Supervised Learning

Bowen Tian†, Songning Lai†, Lujundong Li, Zhihao Shuai, Runwei Guan, Tian Wu, Yutao Yue~

We introduce Precision-Enhanced Pseudo-Labeling (PEPL), a semi-supervised learning approach for fine-grained image classification that generates and refines pseudo-labels using Class Activation Maps (CAMs) to capture essential details, significantly improving accuracy and robustness over existing methods on benchmark datasets. The approach consists of initial and semantic-mixed pseudo-label generation phases to enhance the quality of labels and has been open-sourced for public use.

DRIVE: Dependable Robust Interpretable Visionary Ensemble Framework in Autonomous Driving

Songning Lai†, Tianlang Xue†, Hongru Xiao, Lijie Hu, Jiemin Wu, Ruiqiang Xiao, Ninghui Feng, Haicheng Liao, Zhenning Yang, Yutao Yue~

We introduce DRIVE, a framework designed to enhance the dependability and stability of explanations in end-to-end unsupervised autonomous driving models, addressing instability issues and improving trustworthiness through consistent and stable interpretability and output, as demonstrated by empirical evaluations. This framework provides novel metrics for assessing the reliability of concept-based explainable autonomous driving systems, advancing their real-world deployment.

TimeSieve: Extracting Temporal Dynamics through Information Bottleneck

Ninghui Feng†, Songning Lai†, Fobao Zhou, Zhenxiao Yin, Hang Zhao~

This paper introduces TimeSieve, an innovative time series forecasting model that uses wavelet transforms and information bottleneck theory to capture multi-scale features and filter redundant information, significantly improving accuracy and generalization without manual hyperparameter tuning, thus outperforming existing methods in most cases.

FTS: A FRAMEWORK TO FIND A FAITHFUL TIMESIEVE

Songning Lai†, Ninghui Feng†, Jiechao Gao, Hao Wang, Haochen Sui, Xin Zou, Jiayu Yang, Wenshuo Chen, Hang Zhao, Xuming Hu, Yutao Yue~

• Our research provides faithful technical support and theoretical support to the field of time series forecasting, promising to advance the development and reliability of forecasting methods within the industry. Through these efforts, we aim to bolster the trustworthiness of models, ultimately supporting decision-making processes that rely on accurate and consistent predictions.

Maintaining Informative Coherence: Migrating Hallucinations in Large Language Models via Absorbing Markov Chains

Jiemin Wu, Songning Lai, Ruiqiang Xiao, Tianlang Xue, Jiayu Yang, Yutao Yue

We propose a novel decoding strategy using absorbing Markov chains to measure and enhance the fidelity of contextual information in Large Language Models (LLMs), reducing hallucinations without additional training, and demonstrating superior performance in maintaining accuracy on benchmarks like TruthfulQA and FACTOR.