Revegetation in eroded regions worldwide is a promising strategy to reduce erosion-induced sedimentation. However, systematic assessments of sediment yield and source changes after large-scale restoration remain limited. The Chinese Loess Plateau, historically experiencing the highest soil erosion rates recorded globally, has implemented extensive revegetation initiatives (e.g., the 1999 Grain for Green Programme) to address this issue. We conducted a comprehensive review and analysis of sediment yields and sources by integrating data from check dam deposits, serving as a proxy for erosion in small first-order catchments. Results show that annual specific sediment yields ranged from 498 to 28,460 t/km2 per year over recent decades, with gullies contributing approximately 64% of the total yield. Before 1999, high specific sediment yields were widespread. Following the Grain for Green Programme, yields declined significantly by 16% to 65%, with the greatest reductions occurring in the wetter southern areas. Most of this decrease resulted from reduced sediment in gullies, accounting for 88% of the total change. Although factors like vegetation cover, forest, terracing, rainstorms, and slope all influence specific sediment yields, vegetation cover emerged as the dominant controlling factor after the program, overshadowing other variables. These findings clarify the efficacy of revegetation in specifically mitigating gully erosion in headwater catchments, suggesting that future efforts should focus on understanding how water stress related to revegetation influences erosion processes in arid and fragile regions.