The invention generally relates to a modular fishing hook accessory mounting system for removably mounting a fishing accessory on a hook. The system includes a hook, a modular mounting member and a fishing accessory. The hook has a shank portion and an eye shank portion, the eye shank portion being located at one end of the shank portion. The modular mounting member is mounted on the hook for removably receiving a fishing accessory. The fishing accessory is removably mounted on the modular mounting member, the fishing accessory having a recess formed therein for slidably receiving at least a portion of a retaining member for retaining the retaining member and fishing accessory to the mounting member. The fishing accessory may be a jig head that is substantially lead-free, such as being made from a steel that has been powdered, pressed and sintered. Of course, the modular fishing hook accessory mounting system will work with lead jig head that is modified to be removable.