A device (4) for inserting an infusion cannula (12) or a sensor into a body of a patient comprises an insertion needle (41) adapted to be removably arranged in the infusion cannula (12) or sensor, a handle structure (45) connected (451) to the insertion needle, and a needle cover (42). The needle cover (42) is pivotably mounted (424) on the handle structure (45), and is rotatably movable (51) between a first position, where a front part (413) of the insertion needle (413) is located in the needle cover (42), and a second position, where the front part of the insertion needle is exposed.