Single image depth estimation is an algorithm to estimate depth information from a given RGB or grayscale image only. Compared to multi-view based approaches, single image depth estimation cannot utilize correspondences between images or geometric constraints easily. These limitations lead to poor performance compared to multi-view based approaches. In order to overcome these limitations and achieve good performance, it is natural to take learning based algorithm. Recently, Convolutional Neural Networks-based approaches have shown great success in single image depth estimation task. However, there still exist fundamental limitations such as fixed input size and slow training speed in previous methods. In order to overcome these limitations, we propose a new CNN architecture which can deal with arbitrary sized input images. We also utilize newly designed Normalized Cross Correlation-based loss function. Experimental results show that our algorithm is comparable to state-of-the-art approaches despite its simplicity and fast training time.