CASIA OpenIR  > 紫东太初大模型研究中心
AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models
Zhaopeng Gu1,2; Bingke Zhu1,3; Guibo Zhu1,2; Yingying Chen1,3; Ming Tang1,2; Jinqiao Wang1,2,3
Conference NameThe 38th Annual AAAI Conference on Artificial Intelligence
Conference Date2024-2-20至2024-2-27
Publication PlaceMenlo Park
Large Vision-Language Models (LVLMs) such as MiniGPT-4 and LLaVA have demonstrated the capability of understanding images and achieved remarkable performance in various visual tasks. Despite their strong abilities in recognizing common objects due to extensive training datasets, they lack specifc domain knowledge and have a weaker understanding of  localized details within objects, which hinders their effectiveness in the Industrial Anomaly Detection (IAD) task. On the other hand, most existing IAD methods only provide anomaly scores and necessitate the manual setting of thresholds to distinguish between normal and abnormal samples, which restricts their practical implementation. In this paper, we explore the utilization of LVLM to address the IAD problem and propose AnomalyGPT, a novel IAD approach based on LVLM. We generate training data by simulating anomalous images and producing corresponding textual descriptions for each image. We also employ an image decoder to provide
fne-grained semantic and design a prompt learner to finetune the LVLM using prompt embeddings. Our AnomalyGPT eliminates the need for manual threshold adjustments, thus directly assesses the presence and locations of anomalies. Additionally, AnomalyGPT supports multi-turn dialogues and
exhibits impressive few-shot in-context learning capabilities. With only one normal shot, AnomalyGPT achieves the state of-the-art performance with an accuracy of 86.1%, an image level AUC of 94.1%, and a pixel-level AUC of 95.3% on the MVTec-AD dataset.
Indexed ByEI
IS Representative Paper
Sub direction classification图像视频处理与分析
planning direction of the national heavy laboratory视觉信息处理
Paper associated data
Document Type会议论文
Corresponding AuthorGuibo Zhu; Yingying Chen
Affiliation1.Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences,
2.School of Artifcial Intelligence, University of Chinese Academy of Sciences
3.Objecteye Inc.
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Zhaopeng Gu,Bingke Zhu,Guibo Zhu,et al. AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models[C]. Menlo Park:AAAI,2024.
Files in This Item: Download All
File Name/Size DocType Version Access License
1-Anomalygpt Detecti(4846KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Zhaopeng Gu]'s Articles
[Bingke Zhu]'s Articles
[Guibo Zhu]'s Articles
Baidu academic
Similar articles in Baidu academic
[Zhaopeng Gu]'s Articles
[Bingke Zhu]'s Articles
[Guibo Zhu]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Zhaopeng Gu]'s Articles
[Bingke Zhu]'s Articles
[Guibo Zhu]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 1-Anomalygpt Detecting industrial anomalies using large vision-language models.pdf
Format: Adobe PDF
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.