There is a need for developing accurate learning algorithms for analyzing large-scale medical diagnostic, prognostic, and treatment datasets. Success of classifiers like support vector machines lies in employment of best informative features out of a huge noisy feature space. In this work, we employ a hybrid filter– wrapper approach to build high-performance classification models. We tested our algorithms using popular datasets containing clinic-bio-pathological parameters of leukemia, hepatitis, breast cancer, and colon cancer taken from publically available datasets. Our results indicate that the hybrid algorithm can discover informative subsets possessing very high classification accuracy. © Springer India 2015.