Rill erosion is a major soil erosion process caused by water on extensive sloping farmlands and rangelands worldwide, leading to serious soil loss. To enhance comprehension of the correlation between soil erosion and rill erosion, a field laboratory study of soil erosion by rill formation under various natural conditions and tillage practices was implemented, which was conditions prevalent in the Loess Plateau of northwest China. The slope rills were graded following Strahler's stream ordering, and an analysis was conducted to examine the role of rill erosion in the erosion effect of surface roughness. The impacts of surface roughness on sediment yield and discharge volume exhibited similarities to those of rill erosion, where an increase in rainfall intensity and slope gradient converted inhibition into promotion. From the relationships among the amounts of water and soil conservation and rill erosion at all orders, it was found that second-order rill erosion was the optimum indicator of rill erosion when characterizing sediment yield and runoff. This study established the basis for a comprehensive understanding of the mechanism by which surface roughness influences soil erosion in the Loess Plateau.HIGHLIGHTSRills on the slope were graded according to Strahler's stream ordering. Rill erosions of different orders were calculated. Tillage practices affected the rill erosion changed from inhibition to promotion with slope gradient increased. Surface roughness most likely affected the second-order rill erosion on soil erosion.