Medicinal plants are an important element of indigenous medical systems in Viet Nam. These resources are usually regarded as a part of culture's traditional knowledge. One of the prerequisites for any medical recommendation systems and/or applications is accurate identification and classification of medicinal plants. Hence, leveraging technology in automatic classification of these curative herbs has become essential. Unfortunately, building and training a machine learning model from scratch is next to impossible due to the lack of hardware infrastructure and finance support. It painfully restricts the requirements of rapid solutions to deal with the demand. For this purpose, this paper exploits the idea of transfer learning which is the improvement of learning in a new prediction task through the transferability of knowledge from a related prediction task that has already been learned. By utilizing state-of-the-art deep networks re-trained with our collected data, our extensive experiments show that the proposed combination performs perfectly and achieves the classification accuracy of 98.7% within the acceptable training time.