The role of rock and root structures in preferential flow within mountainous terrains remains an underexplored area in hydrological research. This study aimed to assess the influence of these structures on preferential flow in the rocky mountainous regions of North China, utilizing computed tomography and dye tracing experiments. Our key findings indicated that rock fragments content and coverage increase with slope elevation, with mediumsized rock fragments particles being predominant. Conversely, root characteristics, including quantity density, surface area density, and volume density, were found to be more abundant in the lower slope. Additionally, the staining area from dye tracing, as well as the preferential flow length index and maximum infiltration depth, exhibit a decreasing trend with increasing soil depth, while the upper slope showing significantly higher values compared to the middle and lower slopes. The partial least squares path model indicated that rock fragments were the primary drivers of preferential flow, with medium rock fragments having the most substantial impact. A threshold of approximately 6.92% coverage for coarse rock fragments was identified as the limit beyond which preferential flow is inhibited. Overall, this study provides a thorough analysis of preferential flow dynamics, highlighting the greater influence of rock structures compared to root systems in the study region, and offers critical insights for water resource management in semi-arid mountainous areas.