AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models

CASIA OpenIR > 紫东太初大模型研究中心

	AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models
	Zhaopeng Gu1,2 ; Bingke Zhu1,3 ; Guibo Zhu1,2 ; Yingying Chen 1,3; Ming Tang1,2 ; Jinqiao Wang1,2,3
	2024-02-20
会议名称	The 38th Annual AAAI Conference on Artificial Intelligence
会议日期	2024-2-20至2024-2-27
会议地点	VANCOUVER, CANADA
出版地	Menlo Park
出版者	AAAI
摘要	Large Vision-Language Models (LVLMs) such as MiniGPT-4 and LLaVA have demonstrated the capability of understanding images and achieved remarkable performance in various visual tasks. Despite their strong abilities in recognizing common objects due to extensive training datasets, they lack specifc domain knowledge and have a weaker understanding of localized details within objects, which hinders their effectiveness in the Industrial Anomaly Detection (IAD) task. On the other hand, most existing IAD methods only provide anomaly scores and necessitate the manual setting of thresholds to distinguish between normal and abnormal samples, which restricts their practical implementation. In this paper, we explore the utilization of LVLM to address the IAD problem and propose AnomalyGPT, a novel IAD approach based on LVLM. We generate training data by simulating anomalous images and producing corresponding textual descriptions for each image. We also employ an image decoder to provide fne-grained semantic and design a prompt learner to finetune the LVLM using prompt embeddings. Our AnomalyGPT eliminates the need for manual threshold adjustments, thus directly assesses the presence and locations of anomalies. Additionally, AnomalyGPT supports multi-turn dialogues and exhibits impressive few-shot in-context learning capabilities. With only one normal shot, AnomalyGPT achieves the state of-the-art performance with an accuracy of 86.1%, an image level AUC of 94.1%, and a pixel-level AUC of 95.3% on the MVTec-AD dataset.
收录类别	EI
是否为代表性论文	是
七大方向——子方向分类	图像视频处理与分析
国重实验室规划方向分类	视觉信息处理
是否有论文关联数据集需要存交	否
文献类型	会议论文
条目标识符	http://ir.ia.ac.cn/handle/173211/57293
专题	紫东太初大模型研究中心
通讯作者	Guibo Zhu; Yingying Chen
作者单位	1.Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences, 2.School of Artifcial Intelligence, University of Chinese Academy of Sciences 3.Objecteye Inc.
第一作者单位	中国科学院自动化研究所
通讯作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Zhaopeng Gu,Bingke Zhu,Guibo Zhu,et al. AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models[C]. Menlo Park:AAAI,2024.

条目包含的文件		下载所有文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
1-Anomalygpt Detecti（4846KB）	会议论文		开放获取	CC BY-NC-SA	浏览下载