In this paper, we present a framework for extracting foreground objects based on an estimated blur map. The blur is modeled by spectrum components, spatial gradients and variance of mixed light rays. A blurry background can be effectively generated by summing up rays along a path where camera moves. Using the magnified, synthetic blur due to the camera motion, the proposed method optimizes the initial cost, and segments the interested region from its background planes. The detailed analysis establishes a practical foundation for more general situations with a single hand-held camera.