Traffic congestion poses a significant challenge in Bangladesh due to the growing number of vehicles. To tackle the obstacle needs an effective intelligent system that can reduce the traffic congestion. With a vision of building such an intelligent system, this paper presents a study on vehicle classification using the YOLO (You Only Look Once) v8 transfer learning model, customized for Bangladeshi native vehicles. Besides, we propose a transfer learning model-based system that helps to analyse the video footage of vehicle movements from the elevated viewpoints of foot over-bridges. Initially, the Bangladeshi Native Vehicle Image dataset is gathered, processed, and used to train the model. Once the model is trained and evaluated, the model is integrated into the vehicle detection system. The system detects and tracks the vehicles, providing practical traffic volume and movement insights. After the result analysis, We have found a high mean average precision (mAP) of 91.3 % using intersection over union (IoU). The model’s performance enables proactive measures to reduce congestion and optimise traffic flow. To build an efficient transportation network, this system can assist the Bangladesh Road Transport Authority (BRTA) and Bangladesh Police Traffic Division to address the challenges of increasing traffic and enhance traffic management.