What kind of indoor temperatures are you hitting for it to be that cold for you? You probably want to make sure you’re not running low enough temperatures to allow mold to set in, fwiw.

Regardless, if this is something you want to do, have you covered your bases, wearing a bae layer + wool sweater/socks/cap + maybe fingerless gloves?