You would not have been noticeably warmer (assuming temperature was the only challenge).
The problem is the (apparent) lack of insulation under you. Whatever insulation your bag has is (for the most part) negated when being compressed by your body weight. You were cold because you were only insulating half of your body.