Breast cancer is one of the most dangerous cancers with a high mortality rate, especially for women. Early diagnosis and detection are expected to make the treatment process highly effective. To support doctors and minimize clinical errors, this article proposes a BCICG (Breast cancer image caption generator) approach, which creates breast cancer captioning based on medical images by combining Convolutional Neural Network (CNN) and Transformer architectures. To demonstrate the effectiveness, the proposed approach is then trained on different types of datasets. The results are positive, with the highest BLEU from BLEU-1 to BLEU-4 and Rouge-L measurements being 0.574, 0.521, 0.487, 0.466, and 0.664, respectively. Besides, this study also built a dataset of breast ultrasound images collected in Ca Mau Provincial General Hospital, Vietnam, with descriptions of disease signs based on those images and the doctor’s diagnosis results