Objective Demanding psychosocial work characteristics, such as high job demands, can have a detrimental impact on leisure-time physical activity (LTPA), with adverse consequences for employee health and well-being. However, the mechanisms and moderators of this crossover effect are still largely unknown. We therefore aimed to identify and test potential mediating and moderating factors from within and outside the work environment. Based on the previous research, we expected job demands to be negatively related to LTPA through fatigue. In addition, we expected that job control and worktime control would attenuate the relationship between job demands and fatigue. Furthermore, we hypothesized that autonomous exercise motivation and spontaneous action planning would attenuate the relationship between fatigue and LTPA. In addition to these cross-sectional hypotheses, we expected the same effects to predict a change in LTPA in the following year. Methods To investigate these assumptions, a preregistered longitudinal survey study was conducted among a large sample of Dutch employees in sedentary jobs. Participants reported on the constructs of interest in 2017 and 2018 (N = 1189 and 665 respectively) and the resulting data were analyzed using path analyses. Results Our cross-sectional analyses confirm a weak indirect, negative association between job demands and LTPA, via fatigue. However, this finding was not observed in our longitudinal analyses and none of the other hypotheses were confirmed. Conclusion This study shows that, among employees with relatively healthy psychosocial work characteristics (i.e., high job control), the evidence for an impact of these work characteristics on participation in LTPA is limited.