When it is cold and breezy with the windchill in the 20s, do you stay inside by a warm fire? Of course not, not if you are a photographer. You go outside and do portraits. As long as you dress warmly in layers, you will be just fine.
Photo Data: Canon 5D Mark III. Canon EF24-105mm f/4L IS USM lens at 35 mm. 1/1000 sec, f/8, ISO 400.