The present invention relates to a method for producing a powdery flavoring agent, including (1) a step of treating a milk-derived material with at least any one of a lactic acid fermentation treatment and an enzymatic reaction treatment to prepare a paste flavoring agent, (2) a step of adding an excipient to the paste flavoring agent obtained in step (1) to prepare a freeze-drying raw material liquid having a fat content in a dried solid content of 40% by mass or less, a solid content based on the excipient occupying the dried solid content of 70% by mass or less, and a water content of 20% by mass or more and less than 50% by mass, and (3) a step of subjecting the freeze-drying raw material liquid obtained in the step (2) to specific treatments to obtain the powdery flavoring agent.