I spent one night outside, at -28 degrees F, in an -40 arctic bag. I had no tent, only a small poly tarp between the snow and my bag. I stayed fully clothed but barely slept due to being so cold all night.
Would a tent have kept me warmer?
The tent will definitely be warmer because it keeps the wind away. As you are sleeping your body produces heat and a tent helps keep that warmer air around instead of it blowing away in the wind.
With that said, tents don't provide a huge amount of insulation, and so perhaps a better sleeping bag is needed.
Yes, a tent would have kept you warmer
The biggest difference, however, is that you should have used a proper, isolated sleeping pad. You mentioned that there was only a tarp between you and the snow: this means that your sleeping bag was compressed where your body weight was pressing on it and you had almost direct contact with the snow. You need a good isolated pad to combat this, there is no sleeping bag in the world warm enough to prevent that kind of energy loss in direct contact to the ground/snow.
That being said, please note that your sleeping bag rating is to be taken with a grain of salt, especially the lower end 'extreme' range: after all, what does 'extreme' mean? You barely survive but with arms and legs frozen off?
See also this question on How are sleeping bag temperature ratings determined?
You will always be warmer in a shelter of any sort than you will be out against the raw elements of winter.
You would be surprised how much warmth a tent can provide, but they still aren't the best shelter in the winter.
Snow is an incredible insulator, so when there's and abundance of snow, it makes sense to make use of it when making your shelter. When tenting, you want to dig a pit to set your tent in, and pile the snow up around the pit to help break the wind. If it snows while you're sleeping, the snow accumulating on your tent will act as a blanket, and keep your tent that much warmer. This is of course if you have a tent that can hold the extra weight of the snow.
Winter shelters that are much better than a tent include digging a snow cave, piling up a Quincy, constructing an igloo, or making a trench shelter.
When I'm winter camping I usually only bring a tarp with me, which I use to make a trench shelter. They're the easiest and quickest snow shelter to build: Dig a trench big enough for you to lay in; lay big sticks across the trench; lay your tarp over the sticks, then pile snow on top of your tarp. I've spent more than one night at 10 below zero in a trench shelter laying on top of my sleeping bag because it gets so warm inside.
As has been mentioned in other answers, your choice in sleeping pad makes a significant difference in staying warm when sleeping on the ground during the winter.
Assuming you have a men's bag, the -40F rating is probably the limit rating, but it might be the extreme rating. If it is the extreme rating then sleeping in -28F, it does not surprise me that you were cold. Even if it is the limit rating, you could still be cold if you did not have the proper pad on the ground, if it was windy, or most importantly are a below average sleeper in terms of warmth. At those temperatures even a slight difference in basal metabolic rate can result in feeling cold.
You should also consider what you wear in the bag. Additional clothing can compress the bag and therefore decrease warmth. The bag also needs to fit you well. If it is too big or too small, it will be colder. There is also the issue of moisture. If you cloths are wet this can decrease the insulation of the bag. You may want to use a vapor barrier at those temperatures. You do not want to breath into the bag.
Coupling all of those things, a tent will provide some warmth, but if you were uncomfortably cold at-28F, it is not going to make you comfortable at -40F.
I'm taking a different tack. I have slept both in and out of a tent on many winter nights, and I will not use a tent unless there is significant wind or precipitation.
Here's the problem: The tent will keep most of the air from moving, but that air loads up with moisture from your sweat, your breath, and any damp clothing that came in with you. The net result is that you are now insulated with moisture laden air, which doesn't work nearly as well.
You sweat about a litre at night. You exhale something close to that. That moisture has to go somewhere, and at below zero temps, air doesn't hold much of it. You wake up with frost lining the tent, and your sleeping bag full of water vapour saturated air, likely with some amount condensed as frost on the inside of your sleeping bag.
You can make a case for a tent:
A: It's constructed with vents near the top and bottom so that it can act as a chimney. It needs to do several air changes an hour to keep up with the moisture release of the occupants. In this case it's likely no warmer than outside. If you are using it as a storm tent above timberline, you need to be able to control this air exchange.
B: I have used canvas wall tents and a sheet metal stove. These are the cat's meow for winter camping. The floor is always cold, but from knee height up, it's warm and DRY. Lines rigged near the peak turn clammy socks and mittens crisply dry by morning. No more beating the frozen boot to accept your foot in the morning. They don't lend themselves to human powered travel
(Historical note: During the Klondike Gold Rush, dog mushers would buy silk tents. Silk was about a third of the weight of canvas, and it was reasonable to pack a 20 pound silk wall tent, and sheetmetal stove made from a 5 gallon kerosene can. But they were well-to-do. The fare for a passenger riding out from Dawson was $600)
If I am in an area that permits it I will cut a bunch of spruce branches, or dogwood or alder twigs and make a pad about 3 inches thick to put my foamies on. This both makes the foamies better insulators, but also acts as a sink for snow on me, my footwear etc, so I don't lie on it and melt it. If the twig bed can compress some, the foam pad molds to you better, and the breeze has less access to that space between sleeping bag and pad.
The usual limit in winter camping is how good your sleeping pad is. If I'm anticipating really cold temps, I will bring two sleeping bags, one sized to fit over the other -- the outer often just a cheap box cut summerweight bag. I will also bring two of the half inch thick closed cell foam pads. Most of the condensation happens on the inner surface and inside the outer bag. In the morning strip the outer bag off, and stuff the inner bag quickly to try to expel most of the warm wet air as before it can condense. Compress the outer bag too, but before packing it, shake as much frost off of it as possible.
Insulation underneath is important. I have a knee length heavy duty parka that I used on dog sled expeditions. It was made from nylon pack cloth, with a heavy weight wool blanket cloth lining, and wolf fur rimmed hood. I would use it on top of my foam pad under my sleeping bag. That gave me an extra bit of insulation there. If there was a breeze running at night, I would sleep on one edge of it, and arrange the rest to block the wind.
If there are light breezes I will set up a tarp to block the wind. Preferred location is 45 degrees off the wind, facing the fire. This is the sweet spot for minimum smoke, and maximum wind protection. Heavier winds mean I'm away from the fire, under a spruce tree, with the tarp set to minimize wind speed. This will mean I have more condensation in my bags in the morning, and a less comfortable night. Ditto precip.
But most nights I just set up by the fire. I cut enough wood to keep the fire going all night. this way my nose stays warm and I don't get my bag soggy from condensation.