It is intended to provide a phytotoxicity controlling agent usable as a herbicide for upland farming, which exerts a sufficient herbicidal effect on weeds growing together with upland crops, for example, wheat, rye, barley, oat, corn, sorghum, cotton, soybean, adzuki bean, oilseed rape, beet, upland rice and so on but causes no phytotoxic phenomenon (for example, growth inhibition, growth suppression, tiller inhibition or yellowing) in cultivated plants, comprising a nuclear-substituted benzoic acid represented by the following general formula:wherein R1 represents linear or branched C4-15 alkyl a salt thereof or an alkyl ester thereof.