A growing body of research has examined the effect of aerobic exercise on cognitive function in people with Alzheimer's Disease (AD), but the findings of the available studies were conflicting. The aim of this study was to explore the effect of aerobic exercise on cognitive function in AD patients. Searches were performed in PubMed, Web of Science, and EBSCO databases from the inception of indexing until 12 November 2021. Cochrane risk assessment tool was used to evaluate the methodological quality of the included literature. From 1942 search records initially identified, 15 randomized controlled trials (RCTs) were considered eligible for systematic review and meta-analysis. Included studies involved 503 participants in 16 exercise groups (mean age: 69.2-84 years) and 406 participants (mean age: 68.9-84 years) in 15 control groups. There was a significant effect of aerobic exercise on increasing mini-mental state examination (MMSE) score in AD patients [weighted mean difference (WMD), 1.50 (95% CI, 0.55 to 2.45), p = 0.002]. Subgroup analyses showed that interventions conducted 30 min per session [WMD, 2.52 (95% CI, 0.84 to 4.20), p = 0.003], less than 150 min per week [WMD, 2.10 (95% CI, 0.84 to 3.37), p = 0.001], and up to three times per week [WMD, 1.68 (95% CI, 0.46 to 2.89), p = 0.007] increased MMSE score significantly. In addition, a worse basal cognitive status was associated with greater improvement in MMSE score. Our analysis indicated that aerobic exercise, especially conducted 30 min per session, less than 150 min per week, and up to three times per week, contributed to improving cognitive function in AD patients. Additionally, a worse basal cognitive status contributed to more significant improvements in cognitive function.