An image processing apparatus 1 is composed of an extraction portion 2, a calculation portion 3, a replacement portion 4, and a three-dimensional image generation portion 5, each of which has a plurality of electronic devices including a CPU. The extraction portion 2 extracts an inappropriate pixel included in an input image. The calculation portion 3 calculates replacing information with which pixel information at the extracted inappropriate pixel is to be replaced. The replacement portion 4 outputs a replaced image obtained by replacing the pixel information at the inappropriate pixel with the calculated replacing information. The three-dimensional image generation portion 5 generates a three-dimensional image based on the replaced image.