Nowadays, with the development of technology and the Internet, most households have surveillance cameras to observe everything around the house. Therefore, detecting abnormal human behaviors using videos generated by surveillance cameras has attracted much recent research. This paper focuses on applying the YOLO v4 to build the model detecting abnormal human behaviors, especially detecting fence climbing behaviors. Experimental results on the dataset, including 5340 images extracted from videos, showed that the model obtained the IoU measure of 71% and F1-score measure of 87%.