An image processing apparatus for scanning endoscope includes: a number-of-interpolations determination section that receives detection signals from a detector for sequentially sampling return light from a subject and determines a number of detection signals used in an interpolation process by an interpolation section based on distances between a coordinate position of a predetermined lattice point in pixel data of a raster scan system and sampling coordinate positions of the detection signals around the coordinate position of the predetermined lattice point and the interpolation section that generates pixel data of the predetermined lattice point by using signals of the sampling coordinate positions, wherein a number of the signals is equal to the number of detection signals determined by the number-of-interpolations determination section near the sampling coordinate position corresponding to the coordinate position of the predetermined lattice point.
