综述与专题评论

深度学习与机器视觉在鱼类加工与品质监测中的研究进展

  • 叶东东 ,
  • 徐霞 ,
  • 丁玉庭
展开
  • 1(浙江工业大学 食品科学与工程学院,浙江 杭州,310014)
    2(全省深蓝渔业资源绿色低碳高效开发重点实验室,浙江 杭州,310014)
    3(国家远洋水产品加工技术研发分中心(杭州),浙江 杭州,310014)
第一作者:硕士研究生(丁玉庭教授为通信作者,E-mail:dingyt@zjut.edu.cn)

收稿日期: 2024-02-19

  修回日期: 2024-03-14

  网络出版日期: 2024-12-17

基金资助

浙江省“尖兵”“领雁”研发攻关计划项目(2022C02025);浙江省基础公益研究计划项目(LTGN23C200017)

Research progress on deep learning and machine vision for fish processing and quality monitoring

  • YE Dongdong ,
  • XU Xia ,
  • DING Yuting
Expand
  • 1(College of Food Science and Technology, Zhejiang University of Technology, Hangzhou 310014, China)
    2(Zhejiang Key Laboratory of Green, Low-carbon and Efficient Development of Marine Fishery Resources, Hangzhou 310014 China)
    3(National R&D Branch Center for Pelagic Aquatic Products Processing (Hangzhou), Hangzhou 310014, China)

Received date: 2024-02-19

  Revised date: 2024-03-14

  Online published: 2024-12-17

摘要

在全球渔业产量不断增加和对鱼类品质保障需求提升的背景下,传统的鱼类加工和品质监测方法大多依赖人工操作,这不仅效率低下而且结果的一致性和准确性难以保证,逐渐无法满足现代需求。机器视觉和深度学习技术的结合,提供了一种高效、自动化的方法来提升鱼类加工与品质监测的准确性和效率。该综述概述了机器视觉系统和深度学习在鱼类加工中的应用,包括分类分拣、切割定位、重量估算等方面,并详细介绍了利用高光谱成像、近红外成像、比色传感器和传统成像等方法在品质监测中的最新研究进展,突出了深度学习在提升这些技术识别、分类精度和处理复杂图像数据能力方面的潜力。尽管机器学习技术在单一的加工问题中取得了成功,但面对复杂数据和环境变化时的适应性仍有限,这促使深度学习的相关研究日益受到重视。该文发现当前针对鱼类加工领域的深度学习研究还相对较少,且缺乏能够综合解决鱼类加工和品质监测多重任务的系统性研究。

本文引用格式

叶东东 , 徐霞 , 丁玉庭 . 深度学习与机器视觉在鱼类加工与品质监测中的研究进展[J]. 食品与发酵工业, 2024 , 50(22) : 389 -398 . DOI: 10.13995/j.cnki.11-1802/ts.038903

Abstract

As global fishery outputs grow and demands for fish quality assurance rise, traditional fish processing and quality monitoring methods are increasingly unable to meet modern requirements.The integration of machine vision and deep learning technologies presents an efficient and automated solution to enhance the accuracy and efficiency of fish processing and quality monitoring.This review outlines the applications of machine vision systems and deep learning in fish processing, including tasks such as sorting, cutting, and mass estimation.It delves into the latest research on quality monitoring using hyperspectral imaging, near-infrared imaging, colorimetric sensors, and traditional imaging, emphasizing the potential of deep learning to improve recognition, classification accuracy, and the processing of complex image data.Despite the success of machine learning in addressing specific processing issues, its adaptability to complex data and changing environmental conditions remains limited, underscoring the increasing importance of deep learning research.However, research in the fish processing domain that utilizes deep learning is still relatively sparse, with a notable absence of comprehensive systems capable of addressing multiple processing and monitoring challenges.This study found an urgent need for future research focused on developing integrated systems that could tackle a variety of tasks in fish processing and quality monitoring.Such systems promise not only to improve efficiency and reduce costs but also to ensure product quality through real-time surveillance.

参考文献

[1] WU L L, PU H B, SUN D W. Novel techniques for evaluating freshness quality attributes of fish: A review of recent developments[J]. Trends in Food Science & Technology, 2019, 83:259-273.
[2] NIMBKAR S, AUDDY M, MANOJ I, et al. Novel techniques for quality evaluation of fish: A review[J]. Food Reviews International, 2023, 39(1):639-662.
[3] SUN D W. Hyperspectral imaging for food quality analysis and control[M]. London: Academic, 2010.
[4] MINAHAL Q, MUNIR S, KOMAL W, et al. Global impact of covid-19 on aquaculture and fisheries: A review[J]. International Journal of Fisheries and Aquatic Studies, 2020, 8(6): 42-48.
[5] SABERIOON M, GHOLIZADEH A, CISAR P, et al. Application of machine vision systems in aquaculture with emphasis on fish: State-of-the-art and key issues[J]. Reviews in Aquaculture, 2017, 9(4):369-387.
[6] CIOFFI R, TRAVAGLIONI M, PISCITELLI G, et al. Artificial intelligence and machine learning applications in smart production: Progress, trends, and directions[J]. Sustainability, 2020, 12(2):492.
[7] SCHMIDHUBER J. Deep learning in neural networks: An overview[J]. Neural Networks, 2015, 61:85-117.
[8] ZHOU L, ZHANG C, LIU F, et al. Application of deep learning in food: A review[J]. Comprehensive Reviews in Food Science and Food Safety, 2019, 18(6):1793-1811.
[9] KHAN M I H, SABLANI S S, NAYAK R, et al. Machine learning-based modeling in food processing applications: State of the art[J]. Comprehensive Reviews in Food Science and Food Safety, 2022, 21(2):1409-1438.
[10] ZHU L L, SPACHOS P, PENSINI E, et al. Deep learning and machine vision for food processing: A survey[J]. Current Research in Food Science, 2021, 4:233-249.
[11] LI D L, DU L. Recent advances of deep learning algorithms for aquacultural machine vision systems with emphasis on fish[J]. Artificial Intelligence Review, 2022, 55(5):4077-4116.
[12] ABIODUN O I, JANTAN A, OMOLARA A E, et al. Comprehensive review of artificial neural network applications to pattern recognition[J]. IEEE Access, 2019, 7:158820-158846.
[13] GU J X, WANG Z H, KUEN J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77:354-377.
[14] WECHSLER H. Neural Networks for Perception[M]. Boston: Academic Press, 1992.
[15] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[M]. Lecture Notes in Computer Science. Cham: Springer International Publishing, 2014:818-833.
[16] WANG K, ZHANG L, TANG W J, et al. Asymmetric guerbet reaction to access chiral alcohols[J]. Angewandte Chemie International Edition, 2020, 59(28):11408-11415.
[17] TAN M, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[C]. Proceedings of the 36th International Conference on Machine Learning, PMLR, 2019: 6105-6114.
[18] HONG H M, YANG X L, YOU Z H, et al. Visual quality detection of aquatic products using machine vision[J]. Aquacultural Engineering, 2014, 63:62-71.
[19] BROSNAN T, SUN D W. Improving quality inspection of food products by computer vision—A review[J]. Journal of Food Engineering, 2004, 61(1):3-16.
[20] VITHU P, MOSES J A. Machine vision system for food grain quality evaluation: A review[J]. Trends in Food Science & Technology, 2016, 56:13-20.
[21] SU Y, ZHANG M, MUJUMDAR A S. Recent developments in smart drying technology[J]. Drying Technology, 2015, 33(3):260-276.
[22] BOUZEMBRAK Y, KLÜCHE M, GAVAI A, et al. Internet of things in food safety: Literature review and a bibliometric analysis[J]. Trends in Food Science & Technology, 2019, 94:54-64.
[23] SHORTIS M, HARVEY E, SEAGER J. A review of the status and trends in underwater videometric measurement[C]. Videometrics IX, San Jose, CA, United States: SPIE Conference, 2007, 6491: 1-26.
[24] JIA X X, MA P H, TARWA K, et al. Machine vision-based colorimetric sensor systems for food applications[J]. Journal of Agriculture and Food Research, 2023, 11:100503.
[25] CHENG J H, SUN D W. Hyperspectral imaging as an effective tool for quality analysis and control of fish and other seafoods: Current research and potential applications[J]. Trends in Food Science & Technology, 2014, 37(2):78-91.
[26] LIU D, ZENG X N, SUN D W. Recent developments and applications of hyperspectral imaging for quality evaluation of agricultural products: A review[J]. Critical Reviews in Food Science and Nutrition, 2015, 55(12):1744-1757.
[27] GOWEN A A, O’DONNELL C P, CULLEN P J, et al. Hyperspectral imaging-an emerging process analytical tool for food quality and safety control[J]. Trends in Food Science & Technology, 2007, 18(12):590-598.
[28] LOHUMI S, LEE S, LEE H, et al. A review of vibrational spectroscopic techniques for the detection of food authenticity and adulteration[J]. Trends in Food Science & Technology, 2015, 46(1):85-98.
[29] LIU D, ZENG X A, SUN D W. NIR spectroscopy and imaging techniques for evaluation of fish quality—A review[J]. Applied Spectroscopy Reviews, 2013, 48(8):609-628.
[30] ALAMPRESE C, CASIRAGHI E. Application of FT-NIR and FT-IR spectroscopy to fish fillet authentication[J]. LWT-Food Science and Technology, 2015, 63(1):720-725.
[31] KANG S B, WEBB J A, ZITNICK C L, et al. A multibaseline stereo system with active illumination and real-time image acquisition[C].Proceedings of IEEE International Conference on Computer Vision. Newyork: IEEE, 1995:88-93.
[32] SUBHI M A, MD ALI S H, ISMAIL A G, et al. Food volume estimation based on stereo image analysis[J]. IEEE Instrumentation & Measurement Magazine, 2018, 21(6):36-43.
[33] LIU W B, LYU J Q, WU D, et al. Cutting techniques in the fish industry: A critical review[J]. Foods, 2022, 11(20):3206.
[34] WANG Z F, MIAO Z J, JONATHAN WU Q M, et al. Low-resolution face recognition: A review[J]. The Visual Computer, 2014, 30(4):359-386.
[35] YANG J G, GUO Y H, WANG X L. Feature extraction of hyperspectral images based on deep Boltzmann machine[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(6):1077-1081.
[36] DONG C, LOY C C, HE K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307.
[37] WANG M F, LIU M Y, ZHANG F H, et al. Fast classification and detection of fish images with YOLOv2[C]. 2018 OCEANS-MTS/IEEE Kobe Techno-Oceans (OTO).Newyork: IEEE, 2018:1-4.
[38] MA Y X, ZHANG P F, TANG Y H. Research on fish image classification based on transfer learning and convolutional neural network model[C]. 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD). Newyork: IEEE, 2018:850-855.
[39] ADEYEYE S A O. Traditional fish processing in Nigeria: A critical review[J]. Nutrition & Food Science, 2016, 46(3):321-335.
[40] BULJO J O, GJERSTAD T B. Robotics and Automation in Seafood Processing[M]. Robotics and Automation in the Food Industry. Amsterdam: Elsevier, 2013:354-384.
[41] EINARSDÓTTIR H, GUÓMUNDSSON B, ÓMARSSON V. Automation in the fish industry[J]. Animal Frontiers, 2022, 12(2):32-39.
[42] STORBECK F, DAAN B. Fish species recognition using computer vision and a neural network[J]. Fisheries Research, 2001, 51(1):11-15.
[43] NAKAI M, SATOH H, TAKEDA F. Proposal of an automatic fish sorting system by intelligent image processing[C]. 2008 World Automation Congress.Newyork: IEEE, 2008:1-6.
[44] RACHMATULLAH M N, SUPRIANA I. Low resolution image fish classification using convolutional neural network[C]. 2018 5th International Conference on Advanced Informatics: Concept Theory and Applications (ICAICTA).Newyork: IEEE, 2018:78-83.
[45] ALLKEN V, HANDEGARD N O, ROSEN S, et al. Fish species identification using a convolutional neural network trained on synthetic data[J]. ICES Journal of Marine Science, 2019, 76(1):342-349.
[46] DE SILVA C W, WICKRAMARACHCHI N. An innovative machine for automated cutting of fish[J]. IEEE/ASME Transactions on Mechatronics, 1997, 2(2):86-98.
[47] GAMAGE L B, DE SILVA C W, GOSINE R G. Statistical pattern recognition for cutter positioning in automated fish processing[C]. Proceedings of IEEE Pacific Rim Conference on Communications Computers and Signal Processing.Newyork: IEEE, 1993:786-789.
[48] HYUN S H, LEE S C, KIM K H, et al. Shape, volume prediction modeling and identical weights cutting for frozen fishes[J]. Journal of Korean Institute of Intelligent Systems, 2012, 22(3):294-299.
[49] LEE M. A method for predicting the cutting points using random sample consensus partitioning technique and AI machine vision[J]. International Journal of Membrane Science and Technology, 2023, 10(4):183-191.
[50] XU J L, SUN D W. Computer vision detection of salmon muscle gaping using convolutional neural network features[J]. Food Analytical Methods, 2018, 11(1):34-47.
[51] BARBEDO J G A. A review on the use of computer vision and artificial intelligence for fish recognition, monitoring, and management: 6[J]. Fishes, 2022, 7(6): 335.
[52] BALABAN M O, UNAL SENGÖR G F, GIL SORIANO M, et al. Using image analysis to predict the weight of Alaskan salmon of different species[J]. Journal of Food Science, 2010, 75(3): E157-E162.
[53] FERNANDES A F A, TURRA E M, DE ALVARENGA É R, et al. Deep Learning image segmentation for extraction of fish body measurements and prediction of body weight and carcass traits in Nile Tilapia[J]. Computers and Electronics in Agriculture, 2020, 170:105274.
[54] SABERIOON M, CÍSAŘ P. Automated within tank fish mass estimation using infrared reflection system[J]. Computers and Electronics in Agriculture, 2018, 150:484-492.
[55] ZHANG L, WANG J P, DUAN Q L. Estimation for fish mass using image analysis and neural network[J]. Computers and Electronics in Agriculture, 2020, 173:105439.
[56] FREITAS J, VAZ-PIRES P, CÂMARA J S. Quality index method for fish quality control: Understanding the applications, the appointed limits and the upcoming trends[J]. Trends in Food Science & Technology, 2021, 111:333-345.
[57] SAEED R, FENG H H, WANG X, et al. Fish quality evaluation by sensor and machine learning: A mechanistic review[J]. Food Control, 2022, 137:108902.
[58] SUN D W. Computer Vision Technology for Food Quality Evaluation[M]. 2nd edition. Amsterdam: Elsevier/Academic Press, 2016.
[59] TAPPI S, ROCCULI P, CIAMPA A, et al. Computer vision system (CVS): A powerful non-destructive technique for the assessment of red mullet (Mullus barbatus) freshness[J]. European Food Research and Technology, 2017, 243(12):2225-2233.
[60] ISSAC A, DUTTA M K, SARKAR B. Computer vision based method for quality and freshness check for fish from segmented gills[J]. Computers and Electronics in Agriculture, 2017, 139:10-21.
[61] SHI C, QIAN J P, HAN S, et al. Developing a machine vision system for simultaneous prediction of freshness indicators based on Tilapia (Oreochromis niloticus) pupil and gill color during storage at 4℃[J]. Food Chemistry, 2018, 243:134-140.
[62] HUANG X Y, XU H X, WU L, et al. A data fusion detection method for fish freshness based on computer vision and near-infrared spectroscopy[J]. Analytical Methods, 2016, 8(14):2929-2935.
[63] TAHERI-GARAVAND A, NASIRI A, BANAN A, et al. Smart deep learning-based approach for non-destructive freshness diagnosis of common carp fish[J]. Journal of Food Engineering, 2020, 278:109930.
[64] WU D, SUN D W. Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review: Part I: Fundamentals[J]. Innovative Food Science & Emerging Technologies, 2013, 19:1-14.
[65] WANG B, SUN J F, XIA L M, et al. The applications of hyperspectral imaging technology for agricultural products quality analysis: A review[J]. Food Reviews International, 2023, 39(2):1043-1062.
[66] CHENG J H, SUN D W, ZENG X N, et al. Non-destructive and rapid determination of TVB-N content for freshness evaluation of grass carp (Ctenopharyngodon idella) by hyperspectral imaging[J]. Innovative Food Science & Emerging Technologies, 2014, 21:179-187.
[67] WANG S N, DAS A K, PANG J, et al. Real-time monitoring the color changes of large yellow croaker (Larimichthys crocea) fillets based on hyperspectral imaging empowered with artificial intelligence[J]. Food Chemistry, 2022, 382:132343.
[68] KHOSHNOUDI-NIA S, MOOSAVI-NASAB M. Prediction of various freshness indicators in fish fillets by one multispectral imaging system[J]. Scientific Reports, 2019, 9(1):14704.
[69] MOOSAVI-NASAB M, KHOSHNOUDI-NIA S, AZIMIFAR Z, et al. Evaluation of the total volatile basic nitrogen (TVB-N) content in fish fillets using hyperspectral imaging coupled with deep learning neural network and meta-analysis[J]. Scientific Reports, 2021, 11(1):5094.
[70] DING R, HUANG X Y, HAN F K, et al. Rapid and nondestructive evaluation of fish freshness by near infrared reflectance spectroscopy combined with chemometrics analysis[J]. Analytical Methods, 2014, 6(24):9675-9683.
[71] PAULINE O, CHANG H T, TSAI I L, et al. Intelligent assessment of the histamine level in mackerel (Scomber australasicus) using near-infrared spectroscopy coupled with a hybrid variable selection strategy[J]. LWT, 2021, 145:111524.
[72] SIVERTSEN A H, KIMIYA T, HEIA K. Automatic freshness assessment of cod (Gadus morhua) fillets by Vis/Nir spectroscopy[J]. Journal of Food Engineering, 2011, 103(3):317-323.
[73] ZOTTE A D, OTTAVIAN M, CONCOLLATO A, et al. Authentication of raw and cooked freeze-dried rainbow trout (Oncorhynchus mykiss) by means of near infrared spectroscopy and data fusion[J]. Food Research International, 2014, 60:180-188.
[74] HUANG X Y, XIN J W, ZHAO J W. A novel technique for rapid evaluation of fish freshness using colorimetric sensor array[J]. Journal of Food Engineering, 2011, 105(4):632-637.
[75] MORSY M K, ZÓR K, KOSTESHA N, et al. Development and validation of a colorimetric sensor array for fish spoilage monitoring[J]. Food Control, 2016, 60:346-352.
[76] DOMÍNGUEZ-ARAGÓN A, OLMEDO-MARTÍNEZ J A, ZARAGOZA-CONTRERAS E A. Colorimetric sensor based on a poly(ortho-phenylenediamine-co-aniline) copolymer for the monitoring of Tilapia (Orechromis niloticus) freshness[J]. Sensors and Actuators B: Chemical, 2018, 259:170-176.
[77] GUO L L, WANG T, WU Z H, et al. Portable food-freshness prediction platform based on colorimetric barcode combinatorics and deep convolutional neural networks[J]. Advanced Materials, 2020, 32(45): 2004805.
[78] MA P H, JIA X X, XU W H, et al. Enhancing salmon freshness monitoring with Sol-gel cellulose nanocrystal colorimetric paper sensors and deep learning methods[J]. Food Bioscience, 2023, 56:103313.
文章导航

/