Computer vision for quality grading in fish processing

Abstract

High labour costs, due to the existing technology that still involves a high degree of manually based processing, incur overall high production costs in the fish processing industry. Therefore, a higher degree of automation of processing lines is often desirable, and this strategy has been adopted by the Norwegian fish processing industry to cut-down production costs. In fish processing, despite a slower uptake than in other domains of industry, the use of computer vision as a strategy for automation is beginning to gain the necessary maturity for online grading and evaluation of various attributes related to fish quality. This can enable lower production costs and simultaneously increase quality through more consistent and non-destructive evaluation of the fish products.

This thesis investigates the possibility for automation of fish processing operations by the application of computer vision. The thesis summarises research conducted towards the development of computer vision-based methods for evaluation of various attributes related to whole fish and flesh quality. A brief summary of the main findings is presented here.

By application of computer vision, a method for the inspection of the presence of residual blood in the body cavity of whole Atlantic salmon was developed to determine the adequacy of washing. Inadequate washing of fish after bleeding is quite common in commercial processing plants. By segmenting the body cavity and performing a colour analysis, it was shown that the degree of bleeding correlated well with colour parameters, resulting in correct classification of the fish with residual blood. The developed computer vision-based classifier showed a good agreement with the manual classification of the fish that needed re-washing. The proposed method has potential to automate this type of inspection in fish processing lines.

In addition, a computer vision-based classifier for quality grading of whole Atlantic salmon in different grading classes, as specified by the industrial standard, was developed. In the proposed solution, after segmentation of the salmon from the image scene, with the use of the computer vision techniques, it was possible to extract non-redundant geometrical features describing the size and shape of fish. Based on these features, a classifier was developed for classification of fish into respective grading classes. The average correct rate of classification was in good agreement with the manual labelling, and the method has a potential for grading of Atlantic salmon in fish processing lines.

Regarding fillet grading, a computer vision-based sorting method for Atlantic salmon fillets according to their colour score was developed. The method and classifier/matching algorithm was based on the present industrial standard NS 9402 for evaluation of fillets by colour according to Roche Cards. As a result, fillets or parts of fillets, could be classified into different colour grades. This is important for the industry since different markets tend to have different preferences for fillet colour. This classification method is suitable for online industrial purposes. In addition, the method gives colour evaluation of fresh and smoked fillets in the CIELab space, similar to the L, a, and b values generated by a Minolta Chromameter, for different parts of fillets as well as for the entire fillet. The advantage of the computer vision-based method derives from the flexibility in the choice of the size of the region of interest of the fillet for colour measurement, as opposed to the Chromameter, where the Minolta generated values are obtained by interrogating a very small area of the fillet (8 mm). The method can also be used for detection of colour non-uniformities (discoloration) in both fresh and smoked fillets.

A method for computer vision-based measurements and monitoring of transient 2D and 3D changes in the size and shape of fillets during the rigor process and ice storage was developed. The method successfully measured the size (length, width, area) and shape (roundness) of Atlantic salmon and cod fillets, and monitored changes to these during ice storage with high precision. This was demonstrated by comparison of the exhausted and anesthetized fillets. By laser scanning of the fillet, it was possible to obtain size changes in the height (mm) and area of the fillet in cross-section. The method can be used not only for size and shape analysis of fillets but also for other fish products, both in on-line, as well as off-line conditions as a tool for monitoring 2D/3D size and shape changes. The method can also be used for determination of fillet yield measured in thickness, which is an important parameter for the industry. Together with the colour grading ability, this method can also be used for full feature evaluation and classification of any fish or food product from a single image (colour, size and shape in 2D/3D).

If filleting of fish is done pre-rigor, care should be exercised during colour grading since transient colour changes occur in the post-mortem period. As these changes are more pronounced than those that occur during ice storage, incorrect colour grading can occur. The computer vision method developed for evaluation of colour changes in fillets during rigor, ice storage, and due to effects of perimortem handling stress was considered as the most suitable method for industrial purposes when compared to both the Minolta Chromamater and sensory analysis by a panel.

A computer vision-based method for evaluation of fresh and smoked fillets with respect to bleeding was developed. This form of evaluation is important for the industry as residual blood in fillets may lead to reduced visual acceptance of the product. The method was considered suitable for the purpose of this type of evaluation.

The developed computer vision methods have potential for automation of the mentioned grading operations in the commercial fish processing lines. Application of the proposed solutions would lower the production costs, while simultaneously increasing the quality of the products through a more consistent and non-destructive evaluation of these products.