A method of forming a three-dimensional memory device includes forming at the least one lower level dielectric layer over a semiconductor substrate, forming a buried source line over the least one lower level dielectric layer and over the semiconductor substrate, such that the buried source line is electrically connected to the semiconductor substrate, forming an alternating stack of insulating layers and sacrificial material layers over the buried source line, such that the sacrificial material layers are subsequently replaced with, electrically conductive layers, forming memory openings through the alternating stack by etching through the alternating stack after the buried source line is electrically connected to the semiconductor substrate, and forming memory stack structures in the memory openings. Each memory stack structure includes a vertical semiconductor channel electrically connected to the buried source line and a memory film laterally surrounding the vertical semiconductor channel.