One of the most important problems in visual tracking is how to incrementally update the appearance model because the
appearance of a target object can be easily changed with time when the target is a deformable object or it is moving under varying illumination conditions. To solve these problems, we present a Rao-Blackwellized particle filter (RBPF)-based object tracking algorithm with the adaptive appearance model represented by a Gaussian mixture model (or a mixture of Gaussians model) because a single Gaussian reveals limitations in modeling the target appearance when observations are corrupted by occlusion or the tracking error. We demonstrate the robustness of the proposed method using well-known databases, such as the CAVIAR and the PETS databases