An apparatus and method of generating a three-dimensional (3D) panoramic image, using a single camera. The apparatus may include a frame slit capturing unit to capture a first frame slit in an Nth video frame, and to capture a second frame slit from an (N+2)th video frame, and a 3D image generating unit to generate a left image using the captured first frame slit and the captured second frame slit. Here, the frame slit capturing unit may capture a third frame slit in an (N+1)th video frame, and captures a fourth frame slit in an (N+3)th video frame, and the 3D image generating unit may generate a right image using the captured third frame slit and the captured fourth frame slit.
