PROBLEM TO BE SOLVED: To provide an Al-Mg alloy material for a high pressure hydrogen gas container capable of being applied for members used for long time under a condition in which load is applied to the member in a high pressure hydrogen gas environment, such as a liner of high pressure hydrogen gas tank.SOLUTION: The Al-Mg alloy material contains, by mass%, Mg of 3.0 to 6.0% and the balance aluminum with inevitable impurities, and is subjected to a solid solution treatment for holding the Al-Mg alloy material at a temperature of 400°C or more and a stabilization treatment for holding the solid-solution treated Al-Mg alloy material at a temperature range of 200 to 300°C for 1 hour or more.