In the steep gully slopes below the gully shoulder line, the effect of vegetation coverage on the relationship between erosion and soil organic carbon loss has not been completely clarified. Our in situ rainfall experiments were implemented on plots with five grass coverages on the gully-slopes of the Loess Plateau. Vegetation coverage affected the linear relationship between runoff and sediment. The slope of the fitted line decreased with increasing coverage, meaning that the increase in coverage reduced the sediment transport capacity of flow. High coverage resulted in higher runoff cost for sediment control than medium coverage. The runoff cost for sediment control values at 75% and 90% coverage was 1.69-1.71 times greater than that of 50% coverage. The lower and high thresholds for vegetation cover related to erosion were 35% and 65%, and higher for runoff (lower threshold: 40%; high threshold: 75%). Increasing coverage reduced the total organic carbon loss (TOC) and decreased by 66.9%-99.9%. There was an increasing power function between runoff and organic carbon loss, and between erosion and organic carbon loss with different vegetation coverage (R-2 > 0.95, p < 0.05). Revegetation reduced organic carbon loss mainly by controlling erosion. The interception effect of vegetation on sediment-bound organic carbon (SOC) (67.2%-96.2%) was greater than that on dissolved organic carbon (DOC) (1.56%-75.5%). Sediment was the main carrier of TOC, and SOC accounted for 87.5%-99.6% of TOC, but the increase of coverage reduced the proportion of SOC in TOC. This study provides a basis for reducing sediment in the Yellow River and stabilizing the carbon sinks.