The high-volume low calcium fly ash (HVFA) cement has been successfully applied to the roller compacted concrete, highway base material, and other structural constructions. The HVFA cement based high performance concretes have high workability, acceptable strengths, and superior durability. However, the issues of increase in setting time and decrease in early compressive strength remain. The aim of this investigation is to use the circulating fluidized bed combustion (CFBC) fly ash to improve the mechanical properties, including the compressive strengths, dynamic Young’s modulus (Ed), and dynamic shear modulus (Gd) of the HVFA cement pastes at early ages. The properties of HVFA cement pastes prepared by ordinary Portland cement (OPC) that was partially replaced by Class F fly ash (FFA) with and without sulfate activators (i.e., gypsum or CFBC fly ash) by 50% in weight were explored. The microstructures and the hydration products of the modified HVFA cement pastes were used to explain the improved compressive strengths and dynamic moduli of the specimens. Experimental results showed that the mixture with sulfate activators (i.e., the CFBC fly ash or gypsum) containing 0.81–1.62% SO3 can significantly improve the mechanical properties of the modified HVFA cement pastes at early ages due to more compacted microstructures. The accelerated hydration of tricalcium silicate (C3S) and more precipitated ettringite (AFt) formation of hardened pastes clearly confirm the important role of CFBC fly ash to enhance the mechanical properties of the modified HVFA cement pastes.