Even if it's only for a day or so, you'll need to rebuild your glycogen stores (though many light packable foods are starchy and will top these up quite nicely). That requires carbohydrates, including simple sugars (and not just one type as there are different receptors). Fat stores will also need to be rebuilt, especially over longer periods, but burning glycogen can lead to significant weight loss (as it's accompanied by quite a lot of water in storage); this can make it look like more fat has been burnt than is really the case.*
There's quite a lot of medical literature on this sort of topic; one example I've been reading recently† is Restoration of Muscle Glycogen and Functional Capacity: Role of Post-Exercise Carbohydrate and Protein Co-Ingestion. The title of that paper makes it clear that (in addition to many and varied carbs) protein is required, and in general difference macronutrients are processed through different pathways so a range allows the body to absorb more energy more quickly. That would be particularly important if you've only got a day or two before moving on again.
There's always the factor that sheer quantity is important, and both availability and palatability may not be what you'd like. Sugary drinks are an effective way to increase calorie consumption when you feel full or don't feel like eating large portions. Fatty foods are energy-rich, but they're rather satiating under normal circumstances (which appears to be less of an issue when severely depleted‡). They may however be a good way to end a meal if you find them tempting, even if you feel full easily.
In general, avoiding foods that are very high in water content and fibre is a good idea. This isn't the time for soup or salad. When trying to get patient's weights back up hospitals have guidelines that include "full cream milk", "extra nutritious snacks (e.g. cheese and biscuits)", and "add grated cheese". It's not in the source I've linked, but I've also seen recommendations to add extra-rich custard.
If you're only in a town for a few hours, but you know resupply options are good, you can get a head start by consuming some reserve food an hour or two before arriving in town. This assumes you have reserve food that's not being kept strictly for emergency use.
* To the extent that after a hard ride on a hot day people told me I'd lost weight since the previous evening, despite plenty of food and drink.
† This was in relation to some rather tough (for me) distance cycling over 3 days, and is covered in a question at bicycles.se, though with a different emphasis.
‡ The Minnestoa Starvation Experiment during late WWII looked at the effects of prolonged calorie restriction on otherwise healthy individuals. We can't assume that the results of this study apply directly to a calorie debt built up over days rather than months, but they found that during the recovery phase normal satiety cues were suppressed, a conclusion that many of us can agree with after prolonged exercise significantly in excess of calorie intake. Other studies also report that bingeing follows prolonged food shortages.