HiBug2: Efficient and Interpretable Error Slice Discovery for Comprehensive Model Debugging

Paper

Despite the significant success of deep learning models in computer vision, they often exhibit systematic failures on specific data subsets, known as error slices. Identifying and mitigating these error slices is crucial to enhancing model robustness and reliability in real-world scenarios. In this paper, we introduce HiBug2, an automated framework for error slice discovery and model repair. HiBug2 first generates task-specific visual attributes to highlight instances prone to errors through an interpretable and structured process. It then employs an efficient slice enumeration algorithm to systematically identify error slices, overcoming the combinatorial challenges that arise during slice exploration. Additionally, HiBug2 extends its capabilities by predicting error slices beyond the validation set, addressing a key limitation of prior approaches. Extensive experiments across multiple domains, including image classification, pose estimation, and object detection - show that HiBug2 not only improves the coherence and precision of identified error slices but also significantly enhances the model repair capabilities.

Overview

HiBug2 is an automated framework for error slice discovery and repair for vision models. It conducts the following steps for precise and efficient error slice discovery:

1. Visual Attribute & Tag Generation: HiBug2 leverages powerful multimodal large language models (MLLMs) to generate task-specific visual attributes to highlight error-prone visual features. A fine-grained tag set is also generated alongside each attribute.
2. Validation Image Tagging: HiBug2 tags each image in the validation set with the corresponding visual attributes and tags.
3. Slice Enumeration: HiBug2 uses an efficient slice enumeration algorithm to systematically identify error slices. The enumerated slices may have low average accuracies or data proportions, respectively for model debugging and data balancing.
4. Model Repair: HiBug2 conducts targeted data retrieval (based on the enumerated slices) for model fine-tuning.

The steps can be run iteratively for multi-round model repair.

Usage

Six substeps of the HiBug2 framework are implemented in ./steps. Refer to ./demo.ipynb for an example.

BibTeX

@article{chen2025hibug2,
  title={HiBug2: Efficient and Interpretable Error Slice Discovery for Comprehensive Model Debugging},
  author={Chen, Muxi and Zhao, Chenchen and Xu, Qiang},
  journal={arXiv preprint arXiv:2501.16751},
  year={2025}
}