It has recently been demonstrated that leaf recognition systems opened to an exciting challenge for computer vision and machine learning. These systems’ actual benefit depends on the recognition capacity of models in unconstrained environments and application scenarios. In this paper, the authors collect a real-world dataset containing 1700 images with 34 types of medicinal plants for the evaluation of object recognition algorithms. The images have been taken in several botanic gardens in much different exposure, distance, and rotation. Then we evaluate several off-the-shelf deep architectures to recognize medicinal plants and take into account the recognition accuracy. An excellent average F1-score of 0.98 is achieved. Finally, we integrate the best approach into our self-developed mobile application that (i) recognizes the medicinal plants in real-time and (ii) proposes their healthcare’s uses and remedies.