It has recently been demonstrated that food recognition systems opened to an exciting challenge for computer vision and machine learning. These systems’ actual benefit depends on the recognition capacity of models in unconstrained environments and application scenarios. In this paper, the authors collect a real-world dataset for the evaluation of object recognition algorithms. The images have been taken in a real bakery shop with cakes arranged in many different ways on a tray. Each tray can have zero or many cakes. The authors have collected a set of 1289 bakery trays for a total of 16 different categories. Then we evaluate several off-the-shelf deep architectures to recognize pastry tray and take into account the recognition accuracy and operation time. Excellent accuracy of 100% is achieved within 20 frames per second. Finally, we integrate the best approach into our self-developed website that (i) recognizes the cakes on the tray and (ii) makes the invoice. This work is the first regarding our collected dataset and the application scenario.