A comprehensive review on the structural evolution, stability, and electronic properties of a series of small neutral silver and gold clusters containing 2–20 atoms is presented. Of the clusters Mn with M = Ag, Au, and n = 2–20 considered, the sizes of Au6, Ag8, Ag20, and Au20 are found to be magic as they hold the numbers of valence electrons corresponding to the closed electron shells 1S2/1P6/2S2/1D10. Their stability pattern and electronic structure can thus be understood on the basis of the phenomenological shell model (PSM). Some basic thermochemical parameters, i.e., binding energies per atom, one-step fragmentation energies, and stepwise dissociation energies, are determined by means of density functional theory (DFT) calculations. Energetic parameters of these species typically exhibit odd-even oscillations. While the absorption spectra of gold clusters are normally characterized by strong optical responses in the UV-Vis range with several well-separated peaks, those of silver clusters are dominated by relatively broad peaks, accompanied with some lower-intensity absorption bands. The magic clusters exhibit simpler absorption spectra that are characterized by dominant broad, highly intense transitions. However, the electron shell model appears to be too simple to account for the electronic excitations.