Cervical cancer affects 570,000 women globally and is among the most common causes of cancer-related deaths. Cervical cancer is caused due to the Human Papilloma Virus (HPV) which leads to abnormal growth of cells in the cervix region. Regular testing for HPV in women has helped reduce the death rate in developed countries. However, developing nations are still struggling to provide low-cost solutions due to the lack of affordable medical facilities. The skewed ratio of the oncologists to patients has also aggravated the problem. Motivated by the Deep Learning solutions in Bio-medical imaging, we propose a novel CervixNet methodology which performs image enhancement on cervigrams followed by Segmenting the Region of Interest (RoI) and then classifying the RoI to determine the appropriate treatment. For the classification task, a novel Hierarchical Convolutional Mixture of Experts (HCME) algorithm is proposed. HCME is capable of tackling the problem of overfitting, given that small datasets are an inherent problem in the field of biomedical imaging. Our proposed methodology has outperformed all the existing methodologies on publicly available Intel and Mobile-ODT Kaggle dataset giving an Accuracy of 96.77\% and kappa score of 0.951. Hence, the results obtained validate our approach to provide first level screening at a low cost. © 2019 IEEE.