I probably didn't explain myself very well. You need to have 'some' air moving through the smoker at all times to keep the smoke from becoming stale and/or forming creosote on your food. That's where the experimenting comes in. I would suggest you open one top vent a 'little' and leave it open. Then open one of the bottom vents just enough to give you a steady flow of thin blue smoke out the top vent. Usually you want to make adjustments to the bottom vent(s) and leave the top one open enough to exhaust the smoke. This will also limit the amount of air getting to your wood, and slow the burning quite a bit.
It may take several smokes to find the balance between losing too much heat and venting the smoke, but it can be done. Once you insulate the cabinet the temp will be much more stable.