Ecosystems are not only affected by current climate but are also shaped by antecedent climate through their influences on vegetation growth and environmental conditions. These lagged responses, known as memory effects, can either exacerbate or mitigate the impacts of climate extremes on ecosystem functions. However, the direction, strength and influential duration of memory effects on ecosystem productivity remain poorly understood. Here we implement an interpretable machine-learning framework based on eddy covariance data to model ecosystem gross primary productivity over the period 1995-2020 and further investigate the characteristics of memory effects on positive and negative extremes of ecosystem productivity. Our results show a large contribution from antecedent climate conditions (38.2%) to ecosystem productivity during extremes, with precipitation accounting for 42.2% of the memory effects, followed by temperature (22.1%) and vapour pressure deficit (20.8%). Extreme events conditioned by long-term climatic variations often cause higher productivity losses than short-term extremes, with semi-arid ecosystems exhibiting the largest productivity anomalies and prolonged memory effects. Our results highlight the role of memory effects in regulating carbon flux variations and provide an observation-constrained benchmark for these effects.