If you move the board up, at first it does not get any smaller (well, a bit but that is neglect-able) and hence the leeway of your boat is not effected. Only when the board is almost full up the exposed area is significantly reduced. A far more important effect is that you move the pressure point of the board towards the rudder and in doing so, change the balance of the boat. By moving the board backward (that is what you initially do by turning it “up”) the pressure on the rudder changes. What I always aim for is to have a neutral rudder because any pressure I have to put on the helm is basically slowing the boat down due to the induced drag.
On a beat to windward the CB has to be full down as to move the leading edge of the CB as far forward as possible. This makes the boat point better. I always use full down on a beat, even if it means there is some pressure on the rudder. Moving the crew and setting up the mast and sails correctly should take the pressure of the rudder in this case but that takes some trial and error to get it right.
Pointing in to the wind may help but always use the last bit of momentum to bring the boat back in position to be ready to catch the next gust of wind. On the other hand, moving the rudder (twice) slows the boat and the effect of your actions may be zero or worse than when the boat was kept on a steady course, as Collin suggests.