Assessing groundwater pollution in arid regions remains challenging due to complex hydrogeochemical processes and the limited interpretability and uncertainty awareness of some predictive models. Effective groundwater quality assessment, therefore, requires models that combine accurate prediction with interpretability. This study applied an integrated approach, combining Shapley Additive Explanations (SHAP), Quantile Regression Forests (QRF), and Bayesian classification to evaluate the Water Pollution Index (WPI) in the Neogene aquifer of Eastern Saudi Arabia. Using data from 300 groundwater samples, chromium (Cr), aluminum (Al), and strontium (Sr) were identified as the most influential contributors to WPI. The QRF model achieved an R-2 of 0.855 and a PICP of 82.2%, with a mean interval width of 0.037, offering calibrated predictive intervals for risk assessment. A Bayes classifier, trained on physicochemical features, yielded 77% accuracy in distinguishing between low and moderate pollution classes, supporting probabilistic risk stratification. Classification results showed that 68% of the samples were moderately polluted, while 32% fell into the low pollution category, with no samples exceeding the high-risk threshold. The outcomes reveal the significance of explainable and uncertainty-aware models for guiding groundwater policy, pollutant monitoring, and risk-based assessment in arid environments.