That is why I have switched to a hiking pants, fleece and a breathable dry top. They are almost as good as a full wet suit IMHO. By choosing different combinations I have solutions for every season and weather. Maybe you should have a look at the video’s on the Rooster sails web site? I got their stuff thanks to another Wayfarer sailor and so far I am very pleased with it.
The hikers (or wetsuit) inside out on a raised spinnaker pole for 15 minutes usually gets them dry again provided it’s not raining.
Best gloves I ever had: long sleeved rubber gloves as used in the chemical industry. These gloves are much longer then kitchen gloves and tight (waterproof) around the arms while wide at the wrist and hands. This gives good isolation and keeps my the hands dry and warm even without fleece gloves inside. Kitchen gloves are a poor replacement but work pretty well to. Don’t buy those leather sailing gloves with cut of fingers. Though better at handling ropes, once they get wet they are cold and stay cold for ever.