Personalized medication recommendations aim to suggest a set of medications based on the clinical conditions of a patient. Not only should the patient's diagnosis, procedure, and medication history be considered, but drug-drug interactions (DDIs) must also be taken into account to prevent adverse drug reactions. Although recent studies on medication recommendation have considered DDIs and patient history, personalized disease progression and prescription have not been explicitly modeled. In this work, we proposed FastRx, a Fastformer-based medication recommendation model to capture longitudinality in patient history, in combination with Graph Convolutional Networks (GCNs) to handle DDIs and co-prescribed medications in Electronic Health Records (EHRs). Our extensive experiments on the MIMIC-III dataset demonstrated superior performance of the proposed FastRx over existing state-of-the-art models for medication recommendation. The source code and data used in the experiments are available at https://github.com/pnmthaoct/FastRx.