I understand your reasoning for the firebrick and I'm trying to imagine a way that will let you use the brick, but also be able to remove it.
The bottom is easy.....it just lays there....(I knew a girl like that once...
For the sides, I think if you could get it to fit fairly close, maybe "pin" it in place with something like a nail, or some type of metal wedge so it could be disassembled to move your cooker without the brick falling and breaking.
As for the top....same as the bottom.......just lay the brick on the outside of your firebox on top. It'll still contain the heat just like it would if it was inside the box.