Yes, it is definitely doable. -20°C is only -4°F. The real question is whether it is doable by you at the level of discomfort and hassle you are willing to put up with. Only you can answer that. At best we can point out what the hassles and discomforts will be.
First, your fear of dying of cold in your sleep is silly. You'd have to do something pretty stupid to die of hypothermia, and even that's not going to happen when you're inside the sleeping bag. The main danger will be from frostbite, but that again would largely need stupidity to help it along, although that's easier to do than outright death from hypothermia.
Your tent will be a long term fixed installation you set up once at a time and conditions of your choosing. You can therefore afford a larger and heavier tent with more stuff you bring in once. Definitely get a tent you can stand upright in. That will make changing clothes much quicker and more comfortable. Since you should be able to keep water out of the tent, get a nice down sleeping bag and a few light blankets. The down bag should be rated for most nights, then put the blankets on top for the few unusually cold nights. Since again weight is not really a issue, get a full sleeping bag, not a mummy bag. They are simply more comfortable. Get a good insulating pad, and another two as backup. Get a tent large enough to fit your sleeping bag and something to sit on next to it, like a folding chair. Put something under the legs to spread out the weight to that they don't hurt the tent floor. Get one of those rubber-backed mats people sometimes put just inside their doors and put it just inside your tent. That allows a place to step with boots still on, then you can sit down on the chair with boots still on the mat to take them off. The crud stays on the mat, which you can shake clean by reaching outside after having put on your down hut booties.
Some things are going to be a hassle. At -4°F you want to keep your gloves on whenever possible, but some tasks will be difficult that way. You end up taking your gloves on and off a lot, trading off efficiency with cold fingers. Get a pair of polypro glove liners. They are thin and still allow many tasks to be done, but provide at least a little insulation. Their main advantage is that any metal you touch won't immediately conduct the heet from your hand away. Touching bare metal at -4°F is a good way to get frostbite.
Be prepared for some discomfort no matter what equipment you have. The toughest part will be getting yourself out of the sleeping bag in the morning. You'll really have to will yourself to leave the warm comfort of the bag and get into the air at probably the coldest part of the day. At some point you'll have to change your clothes and get undressed in the process. That's going to be cold. It won't be cold long enough to be any real danger, so it's really a mindset issue to get over. Whether you can or not and are willing to push yourself in that way only you can say.
You say you have access to heated buildings during the day, so it would make things a lot simpler for you if you don't have to deal with cooking and eating at your camp. Perparing food outside in the cold takes a lot longer than in a heated kitchen, severly limits what you can do, and may also risk predator encounters depending on what is around your area in the winter.
