Yup I think a hair dryer is in fact sort of important if you're doing it in the winter.
The thing with breezes is that they disrupt the bird's natural "coccoon" of air around them. So if your bird is fluffed up and trying to build a layer of warm air trapped in their feathers, a breeze blows away that warm air and leaves them cold.
But when the bird is wet, the water is evaporating from their feathers and skin and making them cold - just like when you get goose bumps when you come out of a pool and the air is cold. So that evaporation isn't good. You are trying to help the bird dry off without the evaporation stealing their body warmth, and a gentle warm blow drier does that - gets rid of the water while keeping the bird warm.