PROBLEM TO BE SOLVED: To obtain a hair dye composition which can improve the softness, moistness or smoothness of the hair and is excellent in fastness to dyeing. SOLUTION: This hair dye composition is characterized by comprising a first agent containing an amphoteric polymer compound and/or a cationic polymer compound and a second agent containing one or more kinds selected from among anionic polymer compounds known by the names of Acrylates/Steareth-20 Itaconate Copolymer, Acrylates/Ceteth-20 Itaconate Copolymer, Acrylates/ Steareth-20 Methacrylate Copolymer, Acrylates/Cetech-20 Methacrylate Copolymer, Acrylates/Beheneth-25 Itaconate Copolymer and Acrylates/Beheneth-25 Methacrylate Copolymer.