Grassland plays a key role in global ecological balance. Grazing, being the tool by which humans utilize grasslands, impacts biodiversity. Using a long-term grazing experimental platform in the desert steppe of Inner Mongolia, this study explored the relationships among grazing intensity, plant diversity, soil nutrients, and microbial diversity, and examined variations in plant and soil microbial diversity under four grazing treatments (no grazing, CK; light grazing, LG; moderate grazing, MG; heavy grazing, HG). It also investigated the correlation between soil nutrients and its diversity, indicating that plant diversity (both species and functional diversity) was the highest under LG and MG treatment. Light grazing enhanced the richness indexes of soil microbial communities (bacteria and fungi), while heavy grazing increased bacterial diversity index. Soil nutrient contents exhibited varying degrees of decline with increasing grazing intensity. Compared to the species richness index, the functional dispersion index was more sensitive to grazing intensity. Additionally, soil bacterial diversity was more responsive to grazing intensity than fungal diversity. Soil total nitrogen mediated the relationship between plant and soil microbial diversity, showing positive correlations with both the plant species richness index and the soil microbial Chao1 index. Therefore, rational regulation of grazing intensity to maintain soil nutrient balance may help mitigate desert steppe degradation and enhance the stability of grassland ecosystems.