Plant diversity and aboveground net primary productivity (ANPP) in grasslands are strongly shaped by grazing and modulated by environmental conditions. Yet how moderate grazing regulates plant community dynamics over long timescales, in conjunction with interannual climate variability, remains unclear. Here, using a decade-long moderate grazing experiment in the steppe grasslands of China, we compared grazed grasslands with ungrazed enclosures across interannual variability in precipitation and temperature to evaluate how grazing affects the dynamics of plant diversity, ANPP, and plant functional group composition. Results revealed that livestock grazing reduced ANPP but maintained plant diversity by selectively consuming legumes and annual grasses while conserving other functional groups, resulting in a negative ANPP-plant diversity relationship. Increased precipitation raised plant diversity but lowered ANPP, whereas higher temperatures positively affected ANPP but negatively influenced plant diversity, primarily due to their different roles in regulating interspecific competition in these nutrient-limited grasslands. Notably, moderate grazing intensified the effects of climatic factors on plant communities by enhancing the competitive dominance of less-palatable perennial grasses. Our findings highlight the pivotal role of moderate grazing in shaping plant community dynamics under variable climatic conditions, with implications for biodiversity conservation, productivity, and the optimization of grazing management to sustain grassland ecosystem functioning.