ECML-PKDD 2025 (CCF B) · arXiv · Code (GitHub)

Figure 1: VCT example (OCT2017)

Input image, concept output without perturbation, and concept output after perturbation — illustrating why SVCT is needed for stable explanations.

Abstract

Transparency is a paramount concern in medical AI. Concept Bottleneck Models (CBMs) restrict the latent space to human-understandable concepts, but existing methods often rely solely on concept features for prediction, overlooking intrinsic feature embeddings in medical images, and can be unstable under input perturbations. We propose Vision Concept Transformer (VCT), which uses ViT as the backbone and a label-free conceptual layer, and fuses concept features with image features for decision-making to preserve accuracy while remaining interpretable. We then propose Stable Vision Concept Transformer (SVCT) by integrating Denoised Diffusion Smoothing (DDS) so that the top-k concept indices and predictions remain stable under perturbations, providing faithful explanations. Experiments on four medical datasets (HAM10000, Covid19-CT, BloodMNIST, OCT2017) show that VCT and SVCT maintain accuracy and interpretability, and SVCT provides stable explanations under perturbations.

Figure 2: SVCT framework overview

Overview of the Stable Vision Concept Transformer (SVCT) model.

Method: VCT and SVCT

VCT (Vision Concept Transformer)

• ViT backbone plus a Concept Bottleneck Layer learned in a label-free way (e.g. CLIP-Dissect).
• Concept features f_c(X) are fused with backbone features: F(X) = concat(f(X), W_c f(X)) for the final classifier, avoiding the accuracy drop of concept-only CBMs.

SVCT (Stable VCT)

• Apply Denoised Diffusion Smoothing: add Gaussian noise to token embeddings, then denoise with a diffusion model.
• This yields a stable concept module: (i) explanation stability — top-k concept overlap ≥ β under perturbation; (ii) prediction robustness — Rényi divergence between predictions bounded.
• Default noise level σ = 8/255; evaluation uses perturbation radii ρ_u ∈ [6/255, 10/255].

Experiments (from paper)

Datasets: HAM10000, Covid19-CT, BloodMNIST, OCT2017. Perturbation: PGD radius ρ_u ∈ [6/255, 10/255]; DDS noise σ = 8/255.

Accuracy (with/without perturbation)

Stability: CFS and CPCS

CFS ↓ (lower is better), CPCS ↑ (higher is better). SVCT consistently achieves best stability.

Sensitivity and specificity

Ablation (DDS)

Both denoising and smoothing are essential. Best accuracy and stability (lowest CFS, highest CPCS) when both are enabled. See paper for full ablation tables.

Model configuration & computational cost

Config (paper Table 4): batch_size 512, saga_batch_size 256, proj_batch_size 5000, clip_cutoff 0.25, proj_steps 1000, interpretability_cutoff 0.45, lam 0.0007, n_iters 1000, ρ_u ∈ [6/255, 10/255], S = 8/255.

Cost: ViT 85.8M params / 17.56 GFLOPS; Label-free CBM 85.76M (+40K); SVCT 85.85M (+43K), same GFLOPS.

Visualizations

Figure 3: Concept visualization — one sample per dataset, before vs after perturbation.

Figure 4: Concept-intervention examples — experts can correct concept predictions for co-diagnosis.

Per-dataset concept weight visualizations (one sample each: Covid19-CT, BloodMNIST, HAM10000, OCT2017).

Code & Repo

• GitHub (code, notebooks, data): github.com/xll0328/-ECML-2025-SVCT
• Paper (PDF): arXiv:2506.05286

Citation

@inproceedings{hu2025stable,
  title     = {Stable Vision Concept Transformers for Medical Diagnosis},
  author    = {Hu, Lijie and Lai, Songning and Hua, Yuan and Yang, Shu and Zhang, Jingfeng and Wang, Di},
  booktitle = {Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD)},
  year      = {2025}
}

← Back to Home