Food and Fermentation Industries >

2024 , Vol. 50 >Issue 22: 389 - 398

DOI: https://doi.org/10.13995/j.cnki.11-1802/ts.038903

Research progress on deep learning and machine vision for fish processing and quality monitoring

  • YE Dongdong ,
  • XU Xia ,
  • DING Yuting
Expand
  • 1(College of Food Science and Technology, Zhejiang University of Technology, Hangzhou 310014, China)
    2(Zhejiang Key Laboratory of Green, Low-carbon and Efficient Development of Marine Fishery Resources, Hangzhou 310014 China)
    3(National R&D Branch Center for Pelagic Aquatic Products Processing (Hangzhou), Hangzhou 310014, China)

Received date: 2024-02-19

  Revised date: 2024-03-14

  Online published: 2024-12-17

Fold

Abstract

As global fishery outputs grow and demands for fish quality assurance rise, traditional fish processing and quality monitoring methods are increasingly unable to meet modern requirements.The integration of machine vision and deep learning technologies presents an efficient and automated solution to enhance the accuracy and efficiency of fish processing and quality monitoring.This review outlines the applications of machine vision systems and deep learning in fish processing, including tasks such as sorting, cutting, and mass estimation.It delves into the latest research on quality monitoring using hyperspectral imaging, near-infrared imaging, colorimetric sensors, and traditional imaging, emphasizing the potential of deep learning to improve recognition, classification accuracy, and the processing of complex image data.Despite the success of machine learning in addressing specific processing issues, its adaptability to complex data and changing environmental conditions remains limited, underscoring the increasing importance of deep learning research.However, research in the fish processing domain that utilizes deep learning is still relatively sparse, with a notable absence of comprehensive systems capable of addressing multiple processing and monitoring challenges.This study found an urgent need for future research focused on developing integrated systems that could tackle a variety of tasks in fish processing and quality monitoring.Such systems promise not only to improve efficiency and reduce costs but also to ensure product quality through real-time surveillance.

Key words： fish processing; fish quality; machine vision; deep learning; machine learning

Cite this article

YE Dongdong , XU Xia , DING Yuting . Research progress on deep learning and machine vision for fish processing and quality monitoring[J]. Food and Fermentation Industries, 2024 , 50(22) : 389 -398 . DOI: 10.13995/j.cnki.11-1802/ts.038903

References

[1] WU L L, PU H B, SUN D W. Novel techniques for evaluating freshness quality attributes of fish: A review of recent developments[J]. Trends in Food Science & Technology, 2019, 83:259-273.
[2] NIMBKAR S, AUDDY M, MANOJ I, et al. Novel techniques for quality evaluation of fish: A review[J]. Food Reviews International, 2023, 39(1):639-662.
[3] SUN D W. Hyperspectral imaging for food quality analysis and control[M]. London: Academic, 2010.
[4] MINAHAL Q, MUNIR S, KOMAL W, et al. Global impact of covid-19 on aquaculture and fisheries: A review[J]. International Journal of Fisheries and Aquatic Studies, 2020, 8(6): 42-48.
[5] SABERIOON M, GHOLIZADEH A, CISAR P, et al. Application of machine vision systems in aquaculture with emphasis on fish: State-of-the-art and key issues[J]. Reviews in Aquaculture, 2017, 9(4):369-387.
[6] CIOFFI R, TRAVAGLIONI M, PISCITELLI G, et al. Artificial intelligence and machine learning applications in smart production: Progress, trends, and directions[J]. Sustainability, 2020, 12(2):492.
[7] SCHMIDHUBER J. Deep learning in neural networks: An overview[J]. Neural Networks, 2015, 61:85-117.
[8] ZHOU L, ZHANG C, LIU F, et al. Application of deep learning in food: A review[J]. Comprehensive Reviews in Food Science and Food Safety, 2019, 18(6):1793-1811.
[9] KHAN M I H, SABLANI S S, NAYAK R, et al. Machine learning-based modeling in food processing applications: State of the art[J]. Comprehensive Reviews in Food Science and Food Safety, 2022, 21(2):1409-1438.
[10] ZHU L L, SPACHOS P, PENSINI E, et al. Deep learning and machine vision for food processing: A survey[J]. Current Research in Food Science, 2021, 4:233-249.
[11] LI D L, DU L. Recent advances of deep learning algorithms for aquacultural machine vision systems with emphasis on fish[J]. Artificial Intelligence Review, 2022, 55(5):4077-4116.
[12] ABIODUN O I, JANTAN A, OMOLARA A E, et al. Comprehensive review of artificial neural network applications to pattern recognition[J]. IEEE Access, 2019, 7:158820-158846.
[13] GU J X, WANG Z H, KUEN J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77:354-377.
[14] WECHSLER H. Neural Networks for Perception[M]. Boston: Academic Press, 1992.
[15] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[M]. Lecture Notes in Computer Science. Cham: Springer International Publishing, 2014:818-833.
[16] WANG K, ZHANG L, TANG W J, et al. Asymmetric guerbet reaction to access chiral alcohols[J]. Angewandte Chemie International Edition, 2020, 59(28):11408-11415.
[17] TAN M, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[C]. Proceedings of the 36th International Conference on Machine Learning, PMLR, 2019: 6105-6114.
[18] HONG H M, YANG X L, YOU Z H, et al. Visual quality detection of aquatic products using machine vision[J]. Aquacultural Engineering, 2014, 63:62-71.
[19] BROSNAN T, SUN D W. Improving quality inspection of food products by computer vision—A review[J]. Journal of Food Engineering, 2004, 61(1):3-16.
[20] VITHU P, MOSES J A. Machine vision system for food grain quality evaluation: A review[J]. Trends in Food Science & Technology, 2016, 56:13-20.
[21] SU Y, ZHANG M, MUJUMDAR A S. Recent developments in smart drying technology[J]. Drying Technology, 2015, 33(3):260-276.
[22] BOUZEMBRAK Y, KLÜCHE M, GAVAI A, et al. Internet of things in food safety: Literature review and a bibliometric analysis[J]. Trends in Food Science & Technology, 2019, 94:54-64.
[23] SHORTIS M, HARVEY E, SEAGER J. A review of the status and trends in underwater videometric measurement[C]. Videometrics IX, San Jose, CA, United States: SPIE Conference, 2007, 6491: 1-26.
[24] JIA X X, MA P H, TARWA K, et al. Machine vision-based colorimetric sensor systems for food applications[J]. Journal of Agriculture and Food Research, 2023, 11:100503.
[25] CHENG J H, SUN D W. Hyperspectral imaging as an effective tool for quality analysis and control of fish and other seafoods: Current research and potential applications[J]. Trends in Food Science & Technology, 2014, 37(2):78-91.
[26] LIU D, ZENG X N, SUN D W. Recent developments and applications of hyperspectral imaging for quality evaluation of agricultural products: A review[J]. Critical Reviews in Food Science and Nutrition, 2015, 55(12):1744-1757.
[27] GOWEN A A, O’DONNELL C P, CULLEN P J, et al. Hyperspectral imaging-an emerging process analytical tool for food quality and safety control[J]. Trends in Food Science & Technology, 2007, 18(12):590-598.
[28] LOHUMI S, LEE S, LEE H, et al. A review of vibrational spectroscopic techniques for the detection of food authenticity and adulteration[J]. Trends in Food Science & Technology, 2015, 46(1):85-98.
[29] LIU D, ZENG X A, SUN D W. NIR spectroscopy and imaging techniques for evaluation of fish quality—A review[J]. Applied Spectroscopy Reviews, 2013, 48(8):609-628.
[30] ALAMPRESE C, CASIRAGHI E. Application of FT-NIR and FT-IR spectroscopy to fish fillet authentication[J]. LWT-Food Science and Technology, 2015, 63(1):720-725.
[31] KANG S B, WEBB J A, ZITNICK C L, et al. A multibaseline stereo system with active illumination and real-time image acquisition[C].Proceedings of IEEE International Conference on Computer Vision. Newyork: IEEE, 1995:88-93.
[32] SUBHI M A, MD ALI S H, ISMAIL A G, et al. Food volume estimation based on stereo image analysis[J]. IEEE Instrumentation & Measurement Magazine, 2018, 21(6):36-43.
[33] LIU W B, LYU J Q, WU D, et al. Cutting techniques in the fish industry: A critical review[J]. Foods, 2022, 11(20):3206.
[34] WANG Z F, MIAO Z J, JONATHAN WU Q M, et al. Low-resolution face recognition: A review[J]. The Visual Computer, 2014, 30(4):359-386.
[35] YANG J G, GUO Y H, WANG X L. Feature extraction of hyperspectral images based on deep Boltzmann machine[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(6):1077-1081.
[36] DONG C, LOY C C, HE K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307.
[37] WANG M F, LIU M Y, ZHANG F H, et al. Fast classification and detection of fish images with YOLOv2[C]. 2018 OCEANS-MTS/IEEE Kobe Techno-Oceans (OTO).Newyork: IEEE, 2018:1-4.
[38] MA Y X, ZHANG P F, TANG Y H. Research on fish image classification based on transfer learning and convolutional neural network model[C]. 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD). Newyork: IEEE, 2018:850-855.
[39] ADEYEYE S A O. Traditional fish processing in Nigeria: A critical review[J]. Nutrition & Food Science, 2016, 46(3):321-335.
[40] BULJO J O, GJERSTAD T B. Robotics and Automation in Seafood Processing[M]. Robotics and Automation in the Food Industry. Amsterdam: Elsevier, 2013:354-384.
[41] EINARSDÓTTIR H, GUÓMUNDSSON B, ÓMARSSON V. Automation in the fish industry[J]. Animal Frontiers, 2022, 12(2):32-39.
[42] STORBECK F, DAAN B. Fish species recognition using computer vision and a neural network[J]. Fisheries Research, 2001, 51(1):11-15.
[43] NAKAI M, SATOH H, TAKEDA F. Proposal of an automatic fish sorting system by intelligent image processing[C]. 2008 World Automation Congress.Newyork: IEEE, 2008:1-6.
[44] RACHMATULLAH M N, SUPRIANA I. Low resolution image fish classification using convolutional neural network[C]. 2018 5th International Conference on Advanced Informatics: Concept Theory and Applications (ICAICTA).Newyork: IEEE, 2018:78-83.
[45] ALLKEN V, HANDEGARD N O, ROSEN S, et al. Fish species identification using a convolutional neural network trained on synthetic data[J]. ICES Journal of Marine Science, 2019, 76(1):342-349.
[46] DE SILVA C W, WICKRAMARACHCHI N. An innovative machine for automated cutting of fish[J]. IEEE/ASME Transactions on Mechatronics, 1997, 2(2):86-98.
[47] GAMAGE L B, DE SILVA C W, GOSINE R G. Statistical pattern recognition for cutter positioning in automated fish processing[C]. Proceedings of IEEE Pacific Rim Conference on Communications Computers and Signal Processing.Newyork: IEEE, 1993:786-789.
[48] HYUN S H, LEE S C, KIM K H, et al. Shape, volume prediction modeling and identical weights cutting for frozen fishes[J]. Journal of Korean Institute of Intelligent Systems, 2012, 22(3):294-299.
[49] LEE M. A method for predicting the cutting points using random sample consensus partitioning technique and AI machine vision[J]. International Journal of Membrane Science and Technology, 2023, 10(4):183-191.
[50] XU J L, SUN D W. Computer vision detection of salmon muscle gaping using convolutional neural network features[J]. Food Analytical Methods, 2018, 11(1):34-47.
[51] BARBEDO J G A. A review on the use of computer vision and artificial intelligence for fish recognition, monitoring, and management: 6[J]. Fishes, 2022, 7(6): 335.
[52] BALABAN M O, UNAL SENGÖR G F, GIL SORIANO M, et al. Using image analysis to predict the weight of Alaskan salmon of different species[J]. Journal of Food Science, 2010, 75(3): E157-E162.
[53] FERNANDES A F A, TURRA E M, DE ALVARENGA É R, et al. Deep Learning image segmentation for extraction of fish body measurements and prediction of body weight and carcass traits in Nile Tilapia[J]. Computers and Electronics in Agriculture, 2020, 170:105274.
[54] SABERIOON M, CÍSAŘ P. Automated within tank fish mass estimation using infrared reflection system[J]. Computers and Electronics in Agriculture, 2018, 150:484-492.
[55] ZHANG L, WANG J P, DUAN Q L. Estimation for fish mass using image analysis and neural network[J]. Computers and Electronics in Agriculture, 2020, 173:105439.
[56] FREITAS J, VAZ-PIRES P, CÂMARA J S. Quality index method for fish quality control: Understanding the applications, the appointed limits and the upcoming trends[J]. Trends in Food Science & Technology, 2021, 111:333-345.
[57] SAEED R, FENG H H, WANG X, et al. Fish quality evaluation by sensor and machine learning: A mechanistic review[J]. Food Control, 2022, 137:108902.
[58] SUN D W. Computer Vision Technology for Food Quality Evaluation[M]. 2nd edition. Amsterdam: Elsevier/Academic Press, 2016.
[59] TAPPI S, ROCCULI P, CIAMPA A, et al. Computer vision system (CVS): A powerful non-destructive technique for the assessment of red mullet (Mullus barbatus) freshness[J]. European Food Research and Technology, 2017, 243(12):2225-2233.
[60] ISSAC A, DUTTA M K, SARKAR B. Computer vision based method for quality and freshness check for fish from segmented gills[J]. Computers and Electronics in Agriculture, 2017, 139:10-21.
[61] SHI C, QIAN J P, HAN S, et al. Developing a machine vision system for simultaneous prediction of freshness indicators based on Tilapia (Oreochromis niloticus) pupil and gill color during storage at 4℃[J]. Food Chemistry, 2018, 243:134-140.
[62] HUANG X Y, XU H X, WU L, et al. A data fusion detection method for fish freshness based on computer vision and near-infrared spectroscopy[J]. Analytical Methods, 2016, 8(14):2929-2935.
[63] TAHERI-GARAVAND A, NASIRI A, BANAN A, et al. Smart deep learning-based approach for non-destructive freshness diagnosis of common carp fish[J]. Journal of Food Engineering, 2020, 278:109930.
[64] WU D, SUN D W. Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review: Part I: Fundamentals[J]. Innovative Food Science & Emerging Technologies, 2013, 19:1-14.
[65] WANG B, SUN J F, XIA L M, et al. The applications of hyperspectral imaging technology for agricultural products quality analysis: A review[J]. Food Reviews International, 2023, 39(2):1043-1062.
[66] CHENG J H, SUN D W, ZENG X N, et al. Non-destructive and rapid determination of TVB-N content for freshness evaluation of grass carp (Ctenopharyngodon idella) by hyperspectral imaging[J]. Innovative Food Science & Emerging Technologies, 2014, 21:179-187.
[67] WANG S N, DAS A K, PANG J, et al. Real-time monitoring the color changes of large yellow croaker (Larimichthys crocea) fillets based on hyperspectral imaging empowered with artificial intelligence[J]. Food Chemistry, 2022, 382:132343.
[68] KHOSHNOUDI-NIA S, MOOSAVI-NASAB M. Prediction of various freshness indicators in fish fillets by one multispectral imaging system[J]. Scientific Reports, 2019, 9(1):14704.
[69] MOOSAVI-NASAB M, KHOSHNOUDI-NIA S, AZIMIFAR Z, et al. Evaluation of the total volatile basic nitrogen (TVB-N) content in fish fillets using hyperspectral imaging coupled with deep learning neural network and meta-analysis[J]. Scientific Reports, 2021, 11(1):5094.
[70] DING R, HUANG X Y, HAN F K, et al. Rapid and nondestructive evaluation of fish freshness by near infrared reflectance spectroscopy combined with chemometrics analysis[J]. Analytical Methods, 2014, 6(24):9675-9683.
[71] PAULINE O, CHANG H T, TSAI I L, et al. Intelligent assessment of the histamine level in mackerel (Scomber australasicus) using near-infrared spectroscopy coupled with a hybrid variable selection strategy[J]. LWT, 2021, 145:111524.
[72] SIVERTSEN A H, KIMIYA T, HEIA K. Automatic freshness assessment of cod (Gadus morhua) fillets by Vis/Nir spectroscopy[J]. Journal of Food Engineering, 2011, 103(3):317-323.
[73] ZOTTE A D, OTTAVIAN M, CONCOLLATO A, et al. Authentication of raw and cooked freeze-dried rainbow trout (Oncorhynchus mykiss) by means of near infrared spectroscopy and data fusion[J]. Food Research International, 2014, 60:180-188.
[74] HUANG X Y, XIN J W, ZHAO J W. A novel technique for rapid evaluation of fish freshness using colorimetric sensor array[J]. Journal of Food Engineering, 2011, 105(4):632-637.
[75] MORSY M K, ZÓR K, KOSTESHA N, et al. Development and validation of a colorimetric sensor array for fish spoilage monitoring[J]. Food Control, 2016, 60:346-352.
[76] DOMÍNGUEZ-ARAGÓN A, OLMEDO-MARTÍNEZ J A, ZARAGOZA-CONTRERAS E A. Colorimetric sensor based on a poly(ortho-phenylenediamine-co-aniline) copolymer for the monitoring of Tilapia (Orechromis niloticus) freshness[J]. Sensors and Actuators B: Chemical, 2018, 259:170-176.
[77] GUO L L, WANG T, WU Z H, et al. Portable food-freshness prediction platform based on colorimetric barcode combinatorics and deep convolutional neural networks[J]. Advanced Materials, 2020, 32(45): 2004805.
[78] MA P H, JIA X X, XU W H, et al. Enhancing salmon freshness monitoring with Sol-gel cellulose nanocrystal colorimetric paper sensors and deep learning methods[J]. Food Bioscience, 2023, 56:103313.
Options
Outlines

/

〈 〉

Powered by Beijing Magtech Co. Ltd,.