Multisensor scanner uses AI assistance to detect fruit and vegetable quality
With the global population continuing to grow, food waste not only presents a resource problem but also carries a significant cost factor. A new AI-supported multisensor application developed by the Fraunhofer Cluster of Excellence Cognitive Internet Technologies CCIT is being used to screen perishable goods such as fruit and vegetables. This technology carefully detects quality features even under the surface, making it possible to predict the shelf life of fruit and vegetables. It can be used on entire crates at the touch of a button and customized with the addition of other sensors.
It’s 6:30 a.m. in the logistics hub of a major fruit importer, and a tray containing apples is quickly pushed into the inspection machine. A scale and barcode scanner capture the data from it and compare this with the delivery note. Tapping on the touchscreen also causes near-infrared cameras and microwave radar technology to create an image of the fruit and take a look beneath the peel without destroying any produce.
Using artifical intelligence to predict shelf life
Assisted by AI, a comparison with a stored ideal image of the fruit reveals whether any goods are missing or other varieties have ended up in the tray. It also displays any quality deficiencies on the screen. Using data based on this, it is possible to predict the shelf life of goods and amend the time of sale or pricing as appropriate — making a significant contribution to preventing food waste.
Conserving resources
One of the major aims that Fraunhofer CCIT is incorporating through its new scanner is to optimise the flow of goods and improve planning capability so that, where possible, perishable fresh goods no longer contribute to the approximately 58 million tons of food thrown away every year in the EU alone. This figure also weighs heavily in financial terms, equating to around €132 billion.
Ready for a multitude of applications
Now ready for use, the prototype was developed by the Fraunhofer Institute for Optronics, System Technologies and Image Exploitation IOSB and for Nondestructive Testing IZFP, in conjunction with partners from industry and the food wholesale trade. "This allowed us to accumulate a great deal of practical expertise," explains Anna Taphorn from Fraunhofer IOSB. "The scanner is now ready to be used in other applications." This means additional varieties of fruit and vegetables — and beyond: "We’ve also had a request from the bakery sector, where consistency and freshness are of paramount importance," adds Taphorn, a specialist in visual inspection systems.
Versatile sensors and knowledge transfer opportunities
There are two major aspects contributing to this outstanding versatility. Firstly, the sensor technology is able to adapt to the requirements in question — with gas sensors for detecting ripening gases or hyperspectral cameras that measure sugar content and other internal quality parameters, for instance. Secondly, the type of AI being used offers significant advantages by using anomaly detection to identify issues that would have gone unnoticed before. The model is trained on error-free — that is to say, "ideal" data — and issues a notification when deviations from this data occur. These insights can also be transferred relatively quickly, meaning that only a small quantity of data is required to retrain the model if a new variety of produce is introduced, for example.
Scalability thanks to simple operation
If threshold values are defined for quality features, the AI models are able to make use of recommendations for the next steps that the scanned goods should undergo. This data-centric approach can be multiplied, allowing any number of scanners to be used in parallel. Since the scanners are also easy to operate using a touchpad, they can be used in applications beyond wholesale businesses without the need for costly staff training.
A glimpse of an automated future
"It is possible to envision even more applications in the future. For example, new partnerships could allow us to expand the technology to conveyor belts if required," says Taphorn. This would mean no more need to carry individual crates over to the inspection area manually, plus the ability to use data in a centralised quality check and run entire processes automatically. "This brings huge benefits for speed and quality compared to the manual inspection processes that are commonly used today."
Optimising the flow of goods
Featuring efficient diagnostic capabilities, outstanding versatility and fast availability, the prototype of the multimodal inspection system is now ready to be used in practice. This will open up new horizons, explains Taphorn: "In the future, the customisable sensors and ease of use will play a part in optimizing how fresh goods such as fruit and vegetables are marketed, and in reducing food waste."
The project partners
Fraunhofer IOSB
The Fraunhofer Institute for Optronics, System Technologies and Image Exploitation IOSB is leading the project and is involved in designing and developing the prototype, plus image evaluation using multispectral data.
Fraunhofer IZFP
The Fraunhofer Institute for Nondestructive Testing IZFP has taken charge of integrating and evaluating the microwave sensors, and is making advances in data acquisition and annotation.
Fraunhofer CCIT
In the Fraunhofer Cluster of Excellence Cognitive Internet Technologies CCIT, the Fraunhofer-Gesellschaft deals with cognitive Internet technologies along the entire data processing chain. Fraunhofer CCIT combines the expertise of various Fraunhofer institutes in fields as diverse as microelectronics, information and communication technologies and manufacturing systems. They work together on research and development projects with a focus on the core technologies of IoT communications, trusted data spaces and machine learning.
Source: www.cit.fraunhofer.de
Caption: Data-supported evaluation, versatile applications: The multimodal fruit and vegetable scanner is able to look beneath the peel, assisting in resource-friendly marketing of fresh produce. (Image source: Fraunhofer CIT)