Understanding text that appears in a natural scene is essential to a wide range of applications. This issue is still challenging in the community of document analysis and recognition because of the complexity of the natural scene images. In this paper, we propose a new method to effectively detect text regions by identifying the location of characters. The mainstay of our work is to concentrate on designing a network for text detection and a network for text recognition. For text detection, the proposed method directly predicts characters or text lines that appear in the full scene images, and the approach is able to work for text with arbitrary orientations and quadrilateral shapes. To do that, our model produces the score of character position and the score of character similarity. These scores are utilized to group each character into a single object. For the text recognition phase, the detected text is fed into a second network which is used to extract the features from the text images and to map the features to a sequence of characters. The experiments are performed on public datasets, and the obtained results show that the proposed approach gives competitive performance compared to state-of-the-art approaches.