There are many medical reasons that could contribute to coldness: too thin, anemic, hypothyroidism, poor circulation, tired, dehydrated, not enough vitamin B12, diabetes, lean muscle mass, or maybe you have Raynaud’s. A checkup at the doctor should rule these things out as a cause.
10 Reasons You Feel Cold All The Time
12 Medical Reasons You Always Feel Cold
I would nix most of the clothes, except keep a hat on.
Never wear the day's clothes - if anything, put on tomorrow's clothes.
You say you're wearing wool. Try a synthetic material - polyester - instead. Or silk. Some wools can make you itch, that causes sweating and moisture.
Keep your head outside of the bag: breathing inside the bag brings moisture inside. In this case, your breath is moisturizing your upper body, robbing the lower extremities of heat.
Try using a mummy bag; less air around you to keep warm. Some people find mummies uncomfortable, it takes getting used to.
If you aren't using a mummy, be sure your shoulders are warm; otherwise, you are causing your body to focus on the trunk staying warm, at the expense of the rest of you getting cold.
Try using a softer mat - or a cot. Not for insulation, but for comfort. You may be experiencing hard spots, which cause a feeling of numbness - and cold.
Try sleeping with your head and trunk raised; that will create more blood pressure in the lower extremities.
As for me, I suffer from mild coldness due to joint issues (so, I'm sensitive to hard spots). I use foot warmers - I toss a few into the bag. I also have a Zippo hand warmer (uses Zippo lighter fluid), which lasts about 12-15 hours. These work better with lighter bags for me, since mine keep dying in the middle of the night - I guess maybe because they're starved of air.