Deep learning for multiple retail products image recognition

Taking time to identify expected products and waiting for the check-out in a retail store are common scenes we all encounter in our daily lives. The realisation of automatic product recognition has greatly impacted both commercial and social progress as it is more reliable than manual operation and time-saving. Product recognition via images is a challenging task in the field of computer vision and receives increasing consideration due to the great application prospect, such as automatic check-out, stock tracking, planogram compliance, and visually impaired assistance. In recent years, deep learning has enjoyed a flourishing evolution with tremendous achievements in image classification and object detection. However, the deep learning-based computer vision methods for retail product recognition tasks still lack extensive and in-depth research. This research aims to improve the performance of retail product recognition for the classification and detection tasks.

At first, the multi-angle GAN (MAGAN) for data augmentation is proposed. The MAGAN can be used to produce new single product images with different perspectives to enlarge the training dataset for grocery product recognition. The experimental results on two benchmark datasets (Fruit-360 and Grocery-30) demonstrate that MAGAN outperforms the existing GANs and offers a remarkable recognition result with less training data.

After that, a method that can combine several single-product images into one image is applied. The current image synthesis methods can easily result in a number of objects being completely hidden, as the occlusion correlation between items is not considered. A new instance placement algorithm, named OCA, is developed, which calculates the total hiding area of each item when synthesising check-out images.

The proposed image synthesis approach ensures that no single product is severely obscured in the training images. The current application of CycleGAN for retail product domain adoption does not support the one-to-many translation. This means that we cannot translate the same synthesised check-out image to different domains with corresponding lighting conditions. A novel Image-to-Image translation model (DL-CycleGAN) based on disentangled representation is proposed, which can map a synthetic check-out image to multiple outputs with continuously changing lighting conditions. This model helps reduce the data distribution gap between the training and test sets. Finally, the experimental results indicate that the translated dataset becomes more effective for training the product detection networks.

This thesis reviews the critical challenges of deep learning for retail product recognition and discusses potential techniques that can be helpful for the research of the topic. The MAGAN is proposed for product classification tasks, while OCA and DL-CyclGAN are developed for the multiple products detection tasks in the automatic check-out scenario. This research provides new approaches for improving retail product recognition performance.