The conversion of shrubland to grassland alters primarily the vegetation, but the impact of this conversion on the carbon budget and carbon management in arid and alpine regions remains poorly understood. To fill this research gap, we compared the effects of shrubland-to-grassland conversion, which has been ongoing for at least a decade, on soil organic carbon (SOC) between arid and alpine regions, and examined the mechanisms involved. Shrubland converted to grassland led to a 470% increase in SOC in arid regions; whereas, it led to a 41.4% decrease in SOC in alpine regions. In the conversion to grassland in arid regions, there was a shift in plant species from low- to high-carbon plants, which enhanced carbon input into the soil and facilitated faster decomposition, resulting in an increase in SOC. However, in the conversion to grassland in alpine regions, an opposite trend occurred, resulting in a decrease in SOC. The increase in SOC in arid regions occurred in the late conversion stage; whereas, the decrease in SOC in alpine regions occurred in the early conversion stage. The soil nitrogen to phosphorus ratio (N:P) and the heavy fraction organic carbon (HFOC) were identified as the most important drivers of SOC in arid and alpine regions, respectively, while ecological stoichiometry was the key factor in SOC dynamics. This study provides important insights in SOC dynamics in the conversion of shrubland to grassland. The conversion of shrublands shoud take into account climatic factors, particularly in alpine regions, where shrub removal should be minimized to reduce carbon loss.