You always want to put it into the thickest part of the meat because that is the last to get to the temperature you're looking for. You can measure how deep by holding it on the outside of the meat and estimating what halfway would be, then put your fingers there. When you stick it into the meat for real, stick it in until your fingers are right above the meat. Don't have it touching a bone if there is one.
I'm not going to get into the "how quickly do you put it in" debate because there are people with a whole lot more knowledge about that than me, but that said, you can get around all of the arguments, on bigger hunks of meat anyway, by just being a cheap bastid like me. The longer you wait to put in your probe, the longer your batteries will last!