I remove the bark. Bark usually has a higher percentage of tannins in it. Which could give off a bitter smoke when burned. Oak specifically is one that has a higher amount of tannins in the bark and is used in the tanning process of hides.
Sorry guys, had an early job call in for this morning. Thank you for the replys. I personally don't think it matters too much but I am no expert by any means. And I've only used pecan for the cook as I haven't found anything else in my area yet.