Glyphosate and its main metabolite aminomethylphosphonic acid (AMPA) have frequently been detected in surface water and groundwaters. Since adequate glyphosate mineralization in soil may reduce its losses to environment, improved understanding of site specific factors underlying pesticide mineralization in soils is needed. The aim of this study was to investigate the relationship between soil properties and glyphosate mineralization. To establish a sound basis for resilient correlations, the study was conducted with a large number of 21 agricultural soils, differing in a variety of soil parameters, such as soil texture, soil organic matter content, pH, exchangeable ions etc. The mineralization experiments were carried out with 14C labelled glyphosate at a soil water tension of −15 kPa and at a soil density of 1.3 g cm−3 at 20 ± 1 °C for an incubation period of 32 days. The results showed that the mineralization of glyphosate in different agricultural soils varied to a great extent, from 7 to 70% of the amount initially applied. Glyphosate mineralization started immediately after application, the highest mineralization rates were observed within the first 4 days in most of the 21 soils. Multiple regression analysis revealed exchangeable acidity (H+ and Al3+), exchangeable Ca2+ ions and ammonium lactate extractable K to be the key soil parameters governing glyphosate mineralization in the examined soils. A highly significant negative correlation between mineralized glyphosate and NaOH-extractable residues (NaOH-ER) in soils strongly suggests that NaOH-ER could be used as a simple and reliable parameter for evaluating the glyphosate mineralization capacity. The NaOH-ER were composed of glyphosate, unknown 14C-residues, and AMPA (12%–65%, 3%–34%, 0%–11% of applied 14C, respectively). Our results highlighted the influential role of soil exchangeable acidity, which should therefore be considered in pesticide risk assessments and management to limit efficiently the environmental transfers of glyphosate.