Your solution isn't a bad one.  Adding whatever fleece you have under the raingear would help.  Putting your feet in your pack will help keep them warm.

Much depends on other factors.

 ***How cold is it getting?***  

I have routinely done trips with teenage boys in the eastern slopes of the Canadian Rockies in September.  We have had weather as hot as 30 C (upper 80's) and temps as low as -12 C (~7 F)  We used dacron filled bags nominally rated at -20 C (~0 F)

While it was unusual for a bag to get totally soaked, we found that if it did happen, wring it out, and crawl in. Wear your polypro. It's not a pleasant night, but you will warm up the bag and get some sleep. One person described it as "Making desultory love to a clam" 

I consider September in mountains to be more dangerous than winter. The tendency is for ice water rain and/or sleet, making it harder to keep dry.

***

***Keeping Dry***

We used tarps -- the blue woven plastic ones from Canadian Tire.  Each pair of students or pair of staff were issued with one 10x12 tarp and one 8x10 tarp, and a bunch of light nylon rope.  First night we taught them kids 6 ways to pitch a tarp, depending on anticipated weather and wind.  Then senior boys would check the rigging of the newboys to see they hadn't done something silly like leave part of their ground tarp sticking out where all the drainage from the top tarp would drain onto it. 

Coupled with foam pads, kids generally stayed warm and dry.  Most of the time there was nothing more serious than condensation where a bag was against the plastic.

Now that said:  We were  in areas where fires were permitted.  We cooked on wood, and most evenings built a drying fire. (Do not give me static about fires please. I took my nephew into Willmore last summer in August, the height of the tourist season.  We saw 8 people, all on the last day, 3 hours from the trailhead.  Most trips I've not seen anyone else.)

 This is country with LOTS of creek crossings -- at least one an hour, and on one notable stretch on Mumm pass, 57 crossings in 3 miles. Being well soaked from the knees down, and more than damp above, on a wet day was common.

This meant that a cold wet expedition could stack the wood high to keep warm.  

***Are you above timberline?*** 

  Timberline is associated with two subfactors:  Often little shelter from wind.  No wood for a warming fire.  One of our policies on any trip that had newboys, was that we camped below tree line.  We did alpine camping with the seniors -- mostly, for them it was their third or fourth trip.  They knew how to keep from getting wet.

***Reducing the consequences.***

Ok, you're wet, now what:

* Get out of the wind. If you are in a tent, you've done this, but if you were in a bivy, then you may have the wrong location.  Move to a spot out of hte wind. This may be a small move, or stacking rocks, or putting your pack between you and the wind.
* Wring out your bag.  Get one person on each end and wring it hard.  
* Put on fleece.  It's reasonably warm even wet.   
* Put on socks -- wool or synthetic.
* Wear a toque.
* Eat a thousand calories.  Cheese, peanut butter, nuts.
* Convince your tent mates to let you sleep between them.
* Bring the dog in to curl up beside you.


Contrary to one of the other answers, I would consider moving at night under conditions that got you wet in the first place to be a poor idea.  

* Footing is hard to judge by the light of a flashlight.  If there is a chance that rain will change to snow, it will be even harder.  If it rained,then cleared, you can end up with black ice covered trails.  
* You are moving at night, a time when you are used to sleeping.  Not at your best.
* You are breaking camp at night.  
* Navigation has to be a lot more conscious.  You are 'flying on instruments' You will need your compass and watch more. Or your GPS  
* Everyone has spare batteries for their lights, right?  And will be able to get them out without taking their entire pack apart.

Before you do this in an emergency, do it as practice.