Body part (BP) classification is important in human image analysis. In this paper, we propose an optical flow based pixel-wise BP classifier using random forest (RF) in a monocular video. Running the BP classifier on each optical frame generates noisy mis classifications. We integrate the classifier with a human detector and temporal voting to reduce most of misclassified pixels. The proposed method is evaluated quantitatively on a real world dataset and showed better performance.