Over the past six decades, the implementation of soil and water conservation measures has significantly reduced soil erosion and sediment yield on the Loess Plateau, China. However, while the overall reduction is well-documented, the dynamic interplay between soil erosion potential and sediment connectivity, specifically how they spatially covary under land use/cover changes, remains insufficiently understood. To address this gap, this study established a model framework by integrating the revised universal soil loss equation (RUSLE), index of connectivity (IC), and sediment delivery ratio (SDR) to evaluate the spatio-temporal variations in soil erosion and sediment yield in the Hantaichuan Watershed, northern Loess Plateau, China, from 1995 to 2020 and to estimate the effects of land use/cover changes and check dam construction on sediment yield. The results revealed that the soil erosion in the Hantaichuan Watershed decreased by 43.90% from 1995 to 2020 and the sediment yield decreased by 69.28% under the combination of land use/cover changes and check dam construction. The IC and soil erosion (IC-SE) map revealed both the coupling and decoupling covariation relationships between sediment connectivity and soil erosion. By 2020, areas with high connectivity and high erosion (I-E) covered only 18.67% of the watershed, while contributed more than 40.00% to the total erosion. The I-E zones were mainly located in the central part of the watershed where aeolian sands derived from the Hobq Desert are concentrated and were identified as critical areas for soil and water conservation. This study provides support for priority management of watershed conservation measures as well as a valuable reference for future studies.