Feature selection has received significant attention in knowledge management and decision support systems in the past decades. In this study, kernel-based sparse representation and feature dependence analysis are integrated into a feature assessment and ranking framework. The proposed method utilizes the advantages of the kernel-based sparse representation technique and of the information theoretic metric to iteratively obtain the salient feature cluster. Then, a novel approximate dependence analysis is applied to further maintain complementarity while eliminating redundancy among the features selected by nonlinear orthogonal matching pursuit (NOMP). This can effectively prevent the significant bias caused by the pairwise correlation analysis for a large-scale feature set. To illustrate the effectiveness of the proposed method, classification experiments are conducted with three representative classifiers, on nine well-known datasets. The experimental results show the superiority of the proposed method compared with the representative information theoretic and model-based methods in classification for data-driven decision support systems.