CAFD: Concept-Aware DNN Fault Detection using VLMs
CAFD adds a concept-based signal from vision-language models to improve DNN fault detection while keeping computation practical.
Intelligence analysis by GPT-5.4 Mini
The paper proposes Concept-Aware Fault Detection, a learning-based method that combines model outputs, distance features, and a new concept failure ratio derived from VLMs. In tests across three DNNs and datasets including ImageNet, it outperformed five baselines on fault detection rate.
Imagine a robot that sometimes guesses wrong. This paper teaches a helper to look at the robot’s answers, compare them to other clues, and also read what is in the picture in words.
That extra word-reading clue comes from a vision-language model, which is like a tool that can look at an image and say, “this seems to contain a dog” or “this looks like a car.” The helper then checks whether those clues often show up when the robot makes mistakes.
The paper says this helps catch more errors than older methods, kind of like adding one more smart detective to a team.
Analysis
What CAFD adds
The paper addresses a familiar tradeoff in deep neural network fault detection: stronger hybrid methods can improve results, but they often cost too much to be practical. CAFD, short for Concept-Aware Fault Detection, is presented as a learning-based approach that tries to keep the benefits of multiple information sources without the same computational burden.
How it works
CAFD is trained on a selected feature set rather than on every possible signal. The inputs include model-based signals from the DNN outputs, distance-based features, and a new concept-based feature called Concept Failure Ratio, or CFR. CFR uses vision-language models to extract textual concepts from images and estimate how likely the presence of those concepts is linked to DNN failures. The paper says this gives the detector an extra layer of semantic information.
What the paper reports
The authors say CFR is an effective indicator for DNN fault detection. They evaluate CAFD against five state-of-the-art baselines across three subject DNN models and datasets, including ImageNet. Across constrained selection budgets, CAFD consistently outperformed all baselines in Fault Detection Rate, with an average improvement of 18.3% across the subjects and budget sizes tested.
Why it matters
The result matters because it connects VLMs to a very practical reliability problem: finding when a DNN is likely to fail. If the reported gains hold up, the approach could help make fault detection more effective while staying usable in real deployments.
Key points
- CAFD is a learning-based fault detection method for deep neural networks.
- It combines model outputs, distance features, and a new concept-based signal called Concept Failure Ratio.
- CFR uses vision-language models to extract textual concepts from images and estimate failure likelihood.
- The paper reports gains over five baselines across three DNN subjects and datasets including ImageNet.
- The average Fault Detection Rate improvement reported is 18.3% across the tested settings.
