Foodnet: multi-scale and label dependency learning-based multi-task network for food and ingredient recognition

• 全文链接
• 请求原文

作   者：

Shuang Feng;  Lu Zhouxian;  Li Yong;  Han Chao;  Gu Xia;  Wei Shidi; 

作者机构：

Guangxi University; 

关键词：

Fine-grained recognition;  Food and ingredient recognition;  Multi-task learning; 

期刊名称：

Neural computing & applications

i s s n：

0941-0643

年卷期：

2024 年 36 卷 9 期

页   码：

4485-4501

页   码：

摘   要：

Image-based food pattern classification poses challenges of non-fixed spatial distribution and ingredient occlusion for mainstream computer vision algorithms. However, most current approaches classify food and ingredients by directly extracting abstract features of the entire image through a convolutional neural network (CNN), ignoring the relationship between food and ingredients and ingredient occlusion problem. To address these issues mentioned, we propose a FoodNet for both food and ingredient recognition, which uses a multi-task structure with a multi-scale relationship learning module (MSRL) and a label dependency learning module (LDL). As ingredients normally co-occur in an image, we present the LDL to use the dependency of ingredient to alleviate the occlusion problem of ingredient. MSRL aggregates multi-scale information of food and ingredients, then uses two relational matrixs to model the food-ingredient matching relationship to obtain richer feature representation. The experimental results show that FoodNet can achieve good performance on the Vireo Food-172 and UEC Food-100 datasets. It is worth noting that it reaches the most state-of-the-art level in terms of ingredient recognition in the Vireo Food-172 and UECFood-100.The source code will be made available at https://github.com/visipaper/FoodNet.