A method of manufacturing a semiconductor device includes: forming a doped region in a semiconductor substrate at a first distance to a main surface plane of the semiconductor substrate, wherein the doped region is a first section of a semiconductor column extending from the main surface plane into the semiconductor substrate; forming an insulator structure surrounding at least a second section of the semiconductor column between the main surface plane and the first section in planes parallel to the main surface plane; removing the second section of the semiconductor column; and forming a contact structure extending from the main surface plane to the doped region, wherein the contact structure includes a fill structure and a contact layer, the contact layer formed from a metal semiconductor alloy and directly adjoining the doped region and the fill structure formed from a metal and/or a conductive metal compound.