The device (6) has a piercing tool (1) located in a starting position at a preset piercing angle before piercing. A tool holder (2) is mounted about a pivot axis in a guiding element that is moved up and down by a drive to make pivoting of the tool in relation to the guiding element during engagement with soil (8). An overall centre of gravity of the tool holder and tool is arranged in relation to the axis to partially reduce or compensate or overcompensate moment of inertia of the tool holder and tool so that the moment of inertia forms a part of restoring moment.