The proposed method addresses human action recognition problem in a realistic video. The content of such videos are influenced by irregular background motion and camera shakes. We construct the human pose descriptors by using a modified version of optical flow (we call it as hybrid motion optical flow). We quantize the hybrid motion optical flow (HMOF) into different labels. The orientations of the HMOF vectors are corrected using probabilistic relaxation labelling, where the HMOF vectors with locally maximum magnitude are retained. A sequence of 2D points, called tracks, representing the motion of the person, are constructed. We select top dominant tracks of the sequence based on a cost function. The dominant tracks are further processed to represent the feature descriptor of a given action. © Springer International Publishing Switzerland 2015.