Recommender systems are widely used in many domains, especially in E-commerce. It can be used for attracting users by recommending appropriate products to them. There are many techniques in recommendation systems which can predict rating scores to recommend next products. In this work, we propose an ensemble model for combining Image-based recommendation and Deep Matrix factorization. Specifically, in the proposed model, we have utilized the pre-trained deep learning models (e.g., the VGG16) to extract the image features. Next, based on the image features, we compute similarities between the products to generate recommendations. We integrated the Deep Matrix Factorization model to predict the ratings between users and items into the Image-based recommendation to enhance the effectiveness of the recommendation model. Experimental results on public data sets show that the approach can give good recommendations at more than 90% of accuracy.