An injection device 110 has a removable contact element 220 for pressing against the skin of a patient and which is located at a first end of a sliding sleeve 126 protruding, in its locked position preventing accidental actuation, from an injection device housing 112 and movable towards the housing to an unlocked position. The contact element may grip the skin by the contraction of resilient protrusions 228 formed by a castellated aperture 226, or by a cap (242, figure 5b) acting as a suction cup or by using adhesive (250, figure 6b) on the contact surface. The contact portion is removable from the sliding sleeve and interchangeable with other contact portions, or retrofitted to existing devices.