基于掩模及亮度校正算法的脐橙表面缺陷分割

doi:10.3864/j.issn.0578-1752.2019.02.011

摘要/Abstract

摘要：

【目的】本研究旨在有效解决果皮有缺陷的水果图像在去除背景时部分缺陷被误分割为背景,以及水果表面缺陷难以有效分割提取的问题。【方法】以I分量图来构建掩模模板,根据其灰度直方图信息,通过双峰法选择单一阈值（T=75）分以纽荷尔脐橙为研究对象,提出基于HSI颜色空间模型法去除背景割背景并填充孔洞得到掩模模板I_mask,然后掩模模板I_mask与I分量图通过点乘运算得到去除背景的I分量图;提出基于多尺度高斯函数图像亮度校正算法对去除背景后的I分量图像进行亮度校正,通过构建多尺度高斯函数滤波器,将去除背景后的I分量图与构建的多尺度高斯函数进行卷积运算即得到去除背景后的I分量图像表面光照分量图,最后将去除背景后的I分量图与得到的光照分量图进行点除运算即得到去除背景后的I分量图像亮度校正图;然后采用单一全局阈值法对脐橙表面缺陷进行提取。【结果】基于HSI颜色空间模型法去除背景,可在有效去除背景的同时完好保留脐橙的表面信息,有利于后续操作;基于多尺度高斯函数的图像亮度校正算法分别对6种常见脐橙缺陷进行图像亮度校正后采用单阈值法提取缺陷,使不同灰度等级的脐橙表面缺陷一次性分割成功,其中分割率最高为100%,最低为88.5%,整体达92.7%。通过试验分析后发现造成部分误分割或漏分割的原因主要在于部分缺陷果缺陷处颜色较轻,与正常区域灰度差较小,从而造成漏分割;还有部分缺陷果由于缺陷面积小,在图像形态学处理过程被误认为是噪声而被去除;同时发现正常果的误判率也达到了10.8%,经分析发现误判的正常果表皮组织区域的褶皱位于图像的边缘区域,从而被误认为是边缘区域的缺陷,导致误判。【结论】基于HSI颜色空间模型法去除背景及基于多尺度高斯函数的图像亮度不均校正算法对纽荷尔脐橙图像背景分割和去除背景后的I分量图像表面亮度校正均取得了较好的效果,能有效识别脐橙缺陷区域,为脐橙精确分级提供了技术支持,也为其他果品表面缺陷快速检测提供了一种新思路。

关键词: 脐橙, 表面缺陷, 分割, 去除背景, 亮度校正, 单阈值

Abstract:

【Objective】 The purpose of this study was to effectively solve the problem that some defects of fruit images with defective peels were mistakenly divided into backgrounds when removing background, and it was difficult to effectively segment and extract fruit surface defects.【Method】 Taking Newhall navel orange as the research object, this paper proposed to remove the background based on HSI color space model method to construct the mask template with I component image, and to select a single threshold (T=75) by bimodal method according to its gray histogram information and filled the holes to obtain a mask template. At last, the mask template and I component image were obtained by dot multiplication to obtain I component image from which the background was removed. A multi-scale Gaussian function image brightness correction algorithm was proposed to correct the brightness of I component image after removing the background. By constructing a multi-scale Gaussian function filter, I component image with the background removed and the constructed multi-scale Gaussian function filter were convoluted to obtain the surface illumination component image of I component image after the background was removed. Finally, the I component image after removing the background and the obtained illumination component image were subjected to dot division operation to obtain a luminance correction image of the I component image after the background was removed. At last, the surface defects of navel orange were extracted by a single global threshold method.【Result】 The background was removed based on the HSI color space model method, and the surface information of the navel orange could be preserved while the background was effectively removed, which was beneficial to subsequent operations. The image brightness correction algorithm based on multi-scale Gaussian function was used to extract the defects of the six common navel orange defects, and then the single-threshold method was used to extract the defects. Therefore the surface defects of navel oranges with different gray levels were successfully segmented at one time, and the segmentation rate was up to 100%, the lowest was 88.5%, and the total was 92.7%. Through experimental analysis, it was found that the cause of partial mis-segmentation or leakage segmentation was mainly due to the fact that some defects were lighter in color, and the difference in gray level from normal region was smaller, resulting in leakage segmentation. There were still some defects due to the small defect area, which was mistaken for noise removal during image morphology processing. At the same time, the false positive rate of normal fruit was also found to be 10.8%. It was found that the fold of a part of the normal fruit epidermal tissue area was located in the edge area of the image, which was mistaken for the defect of the edge area, resulting in misjudgment.【Conclusion】 The experimental results showed that image removal based on HSI color space model and image brightness unevenness correction algorithm based on multi-scale Gaussian function had achieved good results for background image segmentation of Newhall navel orange image and I component image surface brightness correction after background removal. It provided technical support for the precise grading of navel oranges and also provided a new idea for the rapid detection of other fruit surface defects.

Key words: navel orange, surface defect, segmentation, remove background, brightness correction, single threshold

张明,李鹏,邓烈,何绍兰,易时来,郑永强,谢让金,马岩岩,吕强. 基于掩模及亮度校正算法的脐橙表面缺陷分割[J]. 中国农业科学, 2019, 52(2): 327-338.

ZHANG Ming,LI Peng,DENG Lie,HE ShaoLan,YI ShiLai,ZHENG YongQiang,XIE RangJin,MA YanYan,LÜ Qiang. Segmentation of Navel Orange Surface Defects Based on Mask and Brightness Correction Algorithm[J]. Scientia Agricultura Sinica, 2019, 52(2): 327-338.

图/表 12

图1

图2

图3

图4

图5

图6

图7

图8

图9

表1

图10

表2

参考文献 35

[1]	VIJAYAREKHA K . Segmentation techniques applied to citrus fruit images for external defect identification. Research Journal of Applied Sciences Engineering & Technology, 2012,4(24):5313-5319.
[2]	ZHANG B H, HUANG W Q, LIANG G, LI J B, ZHAO C J, LIU C L, HUANG D F . Computer vision detection of defective apples using automatic lightness correction and weighted RVM classifier. Journal of Food Engineering, 2015,146:143-151. doi: 10.1016/j.jfoodeng.2014.08.024
[3]	IQBAL S M, GOPAL A, SANKARANARAYANAN P E, NAIR A B . Classification of selected citrus fruits based on color using machine vision system. International Journal of Food Properties, 2016,19(2):272-288. doi: 10.1080/10942912.2015.1020439
[4]	容典 . 脐橙表面缺陷的机器视觉快速检测研究及嵌入式系统应用[D]. 杭州: 浙江大学, 2017.
	RONG D . Study on rapid detection of navel orange surface defect based on machine vision and embedded system application[D]. Hangzhou: Zhejiang University, 2017. ( in Chinese)
[5]	白雪冰, 宋恩来, 李润佳, 许景涛 . 柑橘表面缺陷图像快速准确分割方法. 沈阳农业大学学报, 2018,49(2):242-249.
	BAI X B, SONG E L, LI R J, XU J T . Fast and accurate segmentation method for surface defects of citrus. Journal of Shenyang Agricultural University, 2018,49(2):242-249. (in Chinese)
[6]	周水琴, 应义斌 . 颜色模型在农产品颜色检测与分级中的应用. 浙江大学学报, 2003,29(6):684-688.
	ZHOU S Q, YING Y B . Color models and their applications in color inspection and farm produce grading. Journal of Zhejiang University, 2003,29(6):684-688. (in Chinese)
[7]	THROOP J A, ANESHANSLEY D J, UPCHURCH B L, ANGER B . Apple orientation on two conveyors: Performance and predictability based on fruit shape characteristics. Transactions of the ASAE, 2001,44(1):99-109. doi: 10.13031/2013.2294
[8]	LI J, RAO X, WANG F, WU W, YING Y . Automatic detection of common surface defects on oranges using combined lighting transform and image ratio methods. Postharvest Biology & Technology, 2013,82(4):59-69.
[9]	李江波, 饶秀勤, 应义斌 . 水果表面亮度不均校正及单阈值缺陷提取研究. 农业机械学报, 2011,42(8):159-163.
	LI J B, RAO X Q, YING Y B . Correction algorithm of illumination nonuniformity on fruit surface and defects extraction using single threshold value. Transactions of the Chinese Society of Agricultural Machinery, 2011,42(8):159-163. (in Chinese)
[10]	TAO Y, WEN Z . An adaptive spherical image transform for high-speed fruit defect detection. Transactions of the ASAE, 1999,42(1):241-246. doi: 10.13031/2013.13201
[11]	冯斌, 汪懋华 . 计算机视觉技术识别水果缺陷的一种新方法. 中国农业大学学报, 2002,7(4):73-76.
	FENG B, WANG M H . Study on identifying measurement about default of fruit in computer vision. Journal of China Agricultural University, 2002,7(4):73-76. (in Chinese)
[12]	付峰, 应义斌 . 球体图像灰度变换模型及其在柑桔图像校正中的应用. 农业工程学报, 2004,20(4):117-120.
	FU F, YING Y B . Gray level transform model of ball image and its application in citrus image correction. Transactions of the Chinese Society of Agricultural Engineering, 2004,20(4):117-120. (in Chinese)
[13]	LÓPEZ-GARCÍA F, ANDREU-GARCÍA G, BLASCO J, ALEIXOS N, VALIENTE J M . Automatic detection of skin defects in citrus fruits using a multivariate image analysis approach. Computers & Electronics in Agriculture, 2010,71(2):189-197.
[14]	BLASCO J, ALEIXOS N , GÓMEZ J, MOLTÓ. Citrus sorting by identification of the most common defects using multispectral computer vision. Journal of Food Engineering, 2007,83(3):384-393. doi: 10.1016/j.jfoodeng.2007.03.027
[15]	章海亮, 高俊峰, 何勇 . 基于高光谱成像技术的柑橘缺陷无损检测. 农业机械学报, 2013,44(9):177-181.
	ZHANG H L, GAO J F, HE Y . Nondestructive detection of citrus defection using hyper-spectra imaging technology. Transactions of the Chinese Society of Agricultural Machinery, 2013,44(9):177-181. (in Chinese)
[16]	张保华, 李江波, 樊书祥, 黄文倩, 张驰, 王庆艳, 肖广东 . 高光谱成像技术在果蔬品质与安全无损检测中的原理及应用. 光谱学与光谱分析, 2014,34(10):2743-2751. doi: 10.3964/j.issn.1000-0593(2014)10-2743-09
	ZHANG B H, LI J B, FAN S X, HUANG W Q, ZHANG C, WANG Q Y, XIAO G D . Principles and application of hyperspectral imaging technique in quality and safety inspection of fruits and vegetables. Spectroscopy and Spectral Analysis, 2014,34(10):2743-2751. (in Chinese) doi: 10.3964/j.issn.1000-0593(2014)10-2743-09
[17]	李江波 . 脐橙表面缺陷的快速检测方法研究[D]. 杭州: 浙江大学, 2012.
	LI J B . Study on rapid detection methods of defects on navel orange surface[D]. Hangzhou: Zhejiang University, 2012. ( in Chinese)
[18]	李江波, 饶秀勤, 应义斌, 马本学, 郭俊先 . 基于掩模及边缘灰度补偿算法的脐橙背景及表面缺陷分割. 农业工程学报, 2009,25(12):133-137.
	LI J B, RAO X Q, YING Y B, MA B X, GUO J X . Background and external defects segmentation of navel orange based on mask and edge gray value compensation algorithm. Transactions of the Chinese Society of Agricultural Engineering, 2009,25(12):133-137. (in Chinese)
[19]	李江波, 饶秀勤, 应义斌 . 基于照度-反射模型的脐橙表面缺陷检测. 农业工程学报, 2011,27(7):338-342.
	LI J B, RAO X Q, YING Y B . Detection of navel surface defects based on illumination-reflectance model. Transactions of the Chinese Society of Agricultural Engineering, 2011,27(7):338-342. (in Chinese)
[20]	LAND E H, MCCANN J J . Lightness and Retinex theory. Journal of the Optical Society of America, 1971,61(1):1-11. doi: 10.1364/JOSA.61.000001
[21]	WANG S H, ZHENG J, HU H M, LI B . Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, 2013,22(9):3538-3548. doi: 10.1109/TIP.2013.2261309
[22]	刘志成, 王殿伟, 刘颖, 刘学杰 . 基于二维伽马函数的光照不均匀图像自适应校正算法. 北京理工大学学报, 2016,36(2):191-196.
	LIU Z C, WANG D W, LIU Y, LIU X J . Adaptive adjustment algorithm for non-uniform illumination images based on 2D gamma function. Transaction of Beijing Institute of Technology, 2016,36(2):191-196. (in Chinese)
[23]	王殿伟, 王晶, 许志杰, 刘颖 . 一种光照不均匀图像的自适应校正算法. 系统工程与电子技术, 2017,39(6):1383-1390.
	WANG D W, WANG J, XU Z J, LIU Y . Adaptive correction algorithm for non-uniform illumination images. Journal of Systems Engineering and Electronics, 2017,39(6):1383-1390. (in Chinese)
[24]	饶秀勤 . 基于机器视觉的水果品质实时检测与分级生产线的关键技术研究[D]. 杭州: 浙江大学, 2007.
	RAO X Q . Real-time inspection technology of fruit quality using machine vision[D]. Hangzhou: Zhejiang University, 2007. ( in Chinese)
[25]	白菲 . 基于机器视觉的柑橘水果外形识别方法研究[D]. 北京: 中国农业大学, 2005.
	BAI F . Research on the shape identification of orange based on computer vision technology[D]. Beijing: China Agricultural University, 2005. ( in Chinese)
[26]	应义斌 . 水果图像的背景分割和边缘检测技术研究. 浙江大学学报, 2000,26(1):35-38.
	YING Y B . Study on background segment and edge detection of fruit image using machine vision. Journal of Zhejiang University, 2000,26(1):35-38. (in Chinese)
[27]	XIAO J S, PENG H, ZHANG Y Q, TU C P, LI Q Q . Fast image enhancement based on color space fusion. Color Research & Application, 2016,41(1):22-31.
[28]	姜柏军, 钟明霞 . 改进的直方图均衡化算法在图像增强中的应用. 激光与红外, 2014,44(6):702-706.
	JIANG B J, ZHONG M X . Improved histogram equalization algorithm in the image enhancement. Laser and Infrared, 2014,44(6):702-706. (in Chinese)
[29]	LAND E H . An alternative technique for the computation of the designator in the Retinex theory of color vision. Proceedings of the National Academy of Sciences of the United States of America, 1986,83(10):3078-3080. doi: 10.1073/pnas.83.10.3078
[30]	李福文, 金伟其, 陈伟力, 曹扬, 王霞, 王岭雪 . 基于Retinex模型的彩色图像全局增强算法. 北京理工大学学报, 2010,30(8):947-951.
	LI F W, JIN W Q, CHEN W L, CAO Y, WANG X, WANG L X . Global color image enhancement algorithm based on Retinex model. Transaction of Beijing Institute of Technology, 2010,30(8):947-951. (in Chinese)
[31]	LI J . Application of image enhancement method for digital images based on Retinex theory. Optik-International Journal for Light and Electron Optics, 2013,124(23):5986-5988. doi: 10.1016/j.ijleo.2013.04.115
[32]	赵源萌, 王岭雪, 金伟其, 骆媛 , Saritporn Vittayapadung. 基于区域直方图统计的灰度图像色彩传递方法. 北京理工大学学报, 2012,32(3):322-326.
	ZHAO Y M, WANG L X, JIN W Q, LUO Y, SARITPORN V . A color transfer method for colorization of grayscale image based on region histogram statistics. Transaction of Beijing Institute of Technology, 2012,32(3):322-326. (in Chinese)
[33]	解维域, 丁辰, 宋烨 . GB/T 12947-2008, 鲜柑橘[S]. 北京: 中华人民共和国国家标准化管理委员会, 2008.
	XIE W Y, DING C, SONG Y . GB/T 12947-2008, Fresh citrus[S]. Beijing: Standardization Administration of the People’s Republic of China, 2008. ( in Chinese)
[34]	庞江伟 . 基于计算机视觉的脐橙表面常见缺陷种类识别的研究[D]. 浙江: 浙江大学, 2006.
	PANG J W . Study on external defects classification of navel orange based on machine vision[D]. Hangzhou: Zhejiang University, 2006. ( in Chinese)
[35]	赵杰文, 刘剑华, 陈全胜 , Saritporn Vittayapadung. 利用高光谱图像技术检测水果轻微损伤. 农业机械学报, 2008,39(1):106-109.
	ZHAO J W, LIU J H, CHEN Q S, SARITPORN V . Detecting subtle bruises on fruits with hyperspectral imaging. Transactions of the Chinese Society of Agricultural Machinery, 2008,39(1):106-109. (in Chinese)

算法种类 Algorithm type	亮度校正算法平均耗时 Brightness correction algorithm average time consuming (s)
Retinex算法 Retinex algorithm	0.64
HE算法 HE algorithm	0.53
照度-反射理论 Illumination-reflection theory	2.21
多尺度高斯函数算法 Multiscale Gaussian function algorithm	0.49

表皮类型 Epidermis type	样本数 Number of samples	分割结果 Segmentation result	识别率 Recognition rate (%)
正常果 Normal fruit	65	58	89.2
溃疡果 Canker fruit	49	45	91.8
蓟马果 Thrips fruit	48	43	89.6
油斑病果 Oil spotting fruit	35	31	88.6
黑斑病果 Blackspot fruit	76	74	97.4
风伤果 Wind damage fruit	61	54	88.5
炭疽病果 Anthracnose fruit	66	66	100.0
合计 Total	400	371	92.7