Bayesian Networks (BNs) are effective tools for providing decision support based on expert knowledge in uncertain and complex environments. However, building knowledge-based BNs is still a difficult task that lacks systematic and widely accepted methodologies, especially when knowledge is elicited from multiple experts. We propose a novel method that systematically integrates a widely used Multi Criteria Decision Making (MCDM) approach called Decision Making Trial and Evaluation Laboratory (DEMATEL) in BN construction. Our method elicits causal knowledge from multiple experts based on DEMATEL and transforms it to a BN structure. It then parameterizes the BN by using ranked nodes and evaluates its robustness and consistency by using sensitivity analysis. The proposed method provides a practical and generic way to build probabilistic decision support models by systematically exploiting expert knowledge. Suitable applications of this method include decision problems with multiple criteria, high uncertainty and limited data. We illustrate our method by applying it to a supplier selection case study in a large automobile manufacturer in Turkey. (C) 2019 Elsevier Ltd. All rights reserved.