Free energy is a key thermodynamic observable that controls the elusive physics of the glass transition. However, measuring the free energy of colloidal glasses from microscopy images is challenging due to the difficulty of measuring the individual particle size in the slightly polydisperse glassy systems. In this paper, we carry out experiments and numerical simulations of colloidal glasses with the aim to find a practical approach to measure the free energy from colloidal particles at mild polydispersity. We propose a novel method which requires only the particle coordinates from a few confocal microscopy snapshots to estimate the average particle diameter and use it as an input for our experimental free energy measurements. We verify our free energy calculations from Cell Theory with the free energy obtained by Thermodynamic Integration. The excellent agreement between the free energies measured using the two methods close to the glass transition packing fraction highlights the dominant role played by vibrational entropy in determining a colloidal glass's free energy. Finally, the noticeable free energy difference calculated from uniform and conjectured particle sizes emphasizes the sensitivity on particle free volumes when measuring free energy in the slightly polydisperse colloidal glass.