Recommender systems (RS) are very necessary and important in digital life. Especially, the RS can support users to select appropriate products/items in online systems such as shopping, entertainment, education and other domains. However, techniques in RS are facing with new user problem which means that the RS has no history data to learn and recommend for the users who have not rated the items. This work proposes an approach which is called Content-Based Deep Matrix Factorization (CBDMF) for RS, especially for the new user problem. In this approach, the item information (e.g., item descriptions and other item meta-data) is pre-processed and converted to Term Frequency-Inverse Document Frequency (TF-IDF) vector, then, this vector is integrated with the user and item latent factor vectors before inputting to a deep neuron networks for predictions. We provide architecture of the CBDMF as well as evaluated on several scenarios of new user problems. Experimental results on published Movie and Book data sets show that the CBDMF can work well for recommendations in case of new user problem.