When it comes to processing a deer, most people wash or rinse the meat simply because you are unable to hang it immediately like you would a beef cow once it is killed and drain the blood. Deer usually sit for a a couple of hours before they are hoisted and the blood starts to coagulate. That is why you see more blood in your venison when you thaw it then you do in a store bought piece of beef. The hair of a deer has no bearing on the taste of the animal, unless it is poorly cut and there are a bunch of hairs mixed in with it. The musk gland, or Tarsal gland, is a gland located on the rear legs, behind the first joint. It is a blackish in color and looks like the pad on a dogs paw. This gland is used in marking territory and letting other deer know who's who. This gland always emits an odor, but really starts to stink during the rut, when the buck will bring his hind legs together and forward somewhat, then urinate onto the gland, where his scent is then transferred to the ground for other deer to know he is there. Unless this gland is touched by your hands and you proceed to process your deer, this gland has no impact on the taste of the meat.
As far as a gamy taste, there is no such thing as a gamy taste in wild game. Wild game is WILD game. Whether it be deer, rabbit even fish. Wild games diet is considerably different than that of a pasture or feed lot raised side of beef. It is the difference in diet that gives venison its taste. If you were to eat pen raised venison, you would notice a huge difference in the taste vs. wild venison. When you live on beef your entire life, venison will of course taste completely different than what your used to. If you were raised on venison and someone offered you a t-bone from a cow, that would taste odd to you as you were used to venison.
That is the main reason people say they do not like venison. It simply tastes different and lacks the high fat content of store bought beef.
Now, proper field care is paramount, and can have a huge impact on taste as well. A gut shot deer that has been laying in the woods for a day or two will certainly give the meat in the area of the wound an off taste, as will cutting the bladder and not rinsing it quickly. Game that has taken a bad shot and has run for some time with adrenaline flowing will introduce lactic acid into the muscle tissue, also giving it a off taste, another reason to make sure you practice before going out in the woods.
The key to great tasting wild meat is, prepare ahead of time. Get your hoist ready before your deer arrives to be hung. Make your shot placement so the animal goes down almost immediately. Gut the deer withing minutes of it going down before the blood has a chance to start coagulating. Open the chest cavity and air it out, wiping the inside out the best you can. Use a deer wrap or something to wrap around the deer if it to be pulled out of the woods to keep and dirt and debris off the exposed meat. Cool the carcass as quickly as possible. Hang the carcass out of the sun and if available, give it a hose down with cold water. Hang in temps of 35-40.
Wild game is something that few people will ever enjoy and should be treated for what it is, pure and simple wild game, free of hormones and antibiotics of commercial meat.