This paper presents a real-time background subtraction method which handles illumination changes and dynamic backgrounds such as flapping flags and waving trees. Previous approaches based on Gaussian mixture models usually generates models pixelwise, which makes it difficult to operate in realtime due to computational complexity. Moreover, pixelwise models tend to fail in sudden illumination changes or in dynamic backgrounds. In order to solve this problem, we propose an on-line boosting based background subtraction algorithm. Our approach divides the background area into overlapping patches instead of pixels, and learn classifiers with those patches. The main contribution of this paper is to propose a novel training process for classifiers which use block based Opponent Color Local Binary Pattern (OCLBP). Experimental results show that in environments containing illumination changes and/or dynamic backgrounds, our on-line boosting method using block based OCLBP outperforms previous on-line boosting methods or Gaussian mixture model based methods for robust background subtraction.