A grinding device has a support unit attached to a rotary shaft so as to be rotatable integrally with the rotary shaft, and a grinding element stacked on the support unit and having an uneven grinding surface formed by a plurality of abrasive grains. The support unit has, in a surface thereof, protruding surface portions to be brought into contact with a rear surface of the grinding element when in use, and recessed surface portions recessed from the protruding surface portions, alternately located in a circumferential direction. An engagement mechanism which makes an engagement at each of predetermined angular positions so as to retain the grinding element or the support unit moved by a predetermined angle in the circumferential direction, is provided between the support unit and the grinding element, between the rotary shaft and the grinding element, or between the rotary shaft and the support unit.