Explainable artificial intelligence is increasingly crucial in interpreting deep learning models, particularly in identifying plant diseases. This study proposes a reliability assessment framework using the Focus Score metric by Mosaic Image and the Ablation-CAM technique on a maize leaves disease dataset with fine-tuned MobileNet models. The results show high accuracy in the MobileNetV3 model. However, the reliability of the MobileNetV2 model surpasses in evaluations using the Focus Score metric by Mosaic Image, considering mean, standard deviation, minimum, and maximum values. This demonstrates the success of the proposed framework in thoroughly evaluating black-box models, providing better transparency and effective assessment of saliency maps when ground truth is undetermined and features are hard to distinguish. With these results, future research can use this framework to evaluate various models on training, testing, and validation datasets in a 6:2:2 ratio. Specifically, the Focus Score metric by Mosaic Image can assess reliability, improve accuracy, optimize parameters, and reduce processing time with explainable AI techniques in feature selection.