Depth map up-sampling methods have achieved remarkable improvement by exploiting sensor fusion techniques where they assume that depth map discontinuities and image edges coincide, and the depth values of the temporal neighbors are stable during time variation. However, the assumptions are often violated in the depth data, acquired by active range sensors, due to their inherent noise. This letter presents a new adaptive support method of spatial–temporal neighborhood to alleviate undesirable error propagation, caused by the violations, during depth map sequence up-sampling. On the basis of the spatial–temporal Markov random field model, the proposed method adaptively calculates the weight coefficients of the smoothness term. The experiments show that the proposed method outperforms the previous works in terms of quantitative and qualitative criteria.