Solar irradiance forecasting plays a critical role in the operation and optimization of photovoltaic (PV) systems, battery storage, and hybrid energy systems. Classical forecasting approaches often struggle to capture the intermittent and highly variable nature of solar irradiance, particularly in environments influenced by aerosol loading and dynamic cloud processes. This study evaluates hybrid deep learning architectures combining convolutional feature extraction with recurrent and attention-based sequence modeling for multi-horizon (1-24 h ahead) solar irradiance forecasting. In addition to standard meteorological inputs, particulate matter concentrations (PM10 and PM2.5) are incorporated as exogenous atmospheric variables. The models are trained and validated using multi-year datasets from two contrasting Moroccan environments: a semi-arid site (Benguerir) and a coastal site (Safi). Results show that hybrid CNN-based models outperform classical baselines, and that integrating particulate matter significantly improves forecasting accuracy in semiarid conditions, reducing 1-hour-ahead RMSE by approximately 29-35%. In contrast, at the coastal site, improvements remain limited, indicating that cloud-driven variability dominates irradiance dynamics. Across forecasting horizons, hybrid models demonstrate stable performance with moderate error growth. The results highlight the importance of incorporating site-specific atmospheric variables and provide insight into how forecasting performance depends on the dominant climatic regime.