Our experiment was conducted to investigate the bacterial community structure and the main environmental factors affecting the distribution of endophytic bacteria in the nodules of Vicia villosa. 16S ribosomal RNA (16S rRNA) gene sequence analysis revealed that the community compositions of endophytic bacteria in nodules from four rocky desertification (RD) areas were significantly correlated with the degree of RD. The effects of the physical and chemical properties of soil on community composition were analyzed. The results revealed that (1) the soil water content, falkali-hydrolyzable nitrogen content, and total nitrogen content in the slight rocky desertification (SRD) area GJ4 were significantly greater than those in the moderate rocky desertification (MRD) areas KY2 and MZ3. (2) The composition of the endophytic bacterial community significantly differed among the four groups (R2 = 0.389, P = 0.036). The abundance and diversity of endophytic bacteria in the SRD area were significantly greater than those in the MRD area. In areas of MRD, Pseudomonas was the most abundant genus. The dominant genus in the SRD area was Rhizobium. (3) In Gejiu, Yunnan Province, we observed that higher degrees of rocky desertification were associated with lower soil water content. Additionally, the diversity of endophytic bacteria decreased with increasing rocky desertification. The degree of rocky desertification, total nitrogen content, total phosphorus content, and soil pH are the main factors shaping the community composition of endophytic bacteria in the nodules of V. villosa. This study provides a comprehensive and robust theoretical framework, laying the foundation for the development of effective strategies and measures to control and mitigate the process of rocky desertification.