The edge and motion are the main features that human visual
system (HVS) perceives intensively. Therefore, it is very
important to obtain accurate boundary of moving object
coinciding with the boundary that HVS perceives for humanlike
vision system. This paper proposes an algorithm for the
segmentation of the moving object with accurate boundary
using color and motion focusing on the HVS perception in the
general image sequence. The proposed algorithm is composed
of three parts: color segmentation, motion analysis, and
region refinement and merging part. In the color
segmentation phase, K-Means algorithm is used in
consideration of the sensitivity of the human color perception
to get the accurate boundaries coinciding with the boundaries
that HVS perceives. The global and local motion estimation
are performed in parallel with color analysis. As the result of
color and motion analysis, boundary and motion information
of each region are obtained. After that, Bayesian clustering
using color and motion provides more accurate boundary for
each region although the color contrast between objects and
background is low. In the final stage, regions are merged
taking into account their motion. The experimental results of
the proposed algorithm show the accurate moving object
boundary coinciding with the boundary that HVS perceives.