At my club, we race in a general handicap fleet, containing a huge variety of fast and slow handicap boats. For this, we have ratings for many classes, including different ratings for Wayfarers with and without Spinnakers. I have included the ratings used below, from which you will see that the 4% rule does not apply.
Wayfarer PN 1099
Wayfarer No Spi PN 1109
Not sure if these are official, but if any one has any more accurate/official ratings I would be keenly interested (as a current white sail only racer!).
Kind regards,
Darren