I'm presuming your on an overhanging wall here as if you can touch the wall you should walk down preventing this issue.
It's very common to twist and spin a little when being lowered off overhanging terrain so don't worry about it! Keep calm and relax.
If you tense up and try and correct any spining or swinging your likely only going to make things worse. Do not grab the wall! You may well hurt yourself. Parry it away with your hands and feet but don't hold onto a hold. You could down climb (but this is likely going to become very tiresome very quickly)
I tend to let my arms hang loose by the side of my body and lean forwards slightly. If I spin or swing (providing I'm not going to hit anything) I let it happen and just stay loose. If this becomes too violent, your belayer should slow down or stop, let the swing finish then continue. The belayer should do this without you having to shout and tell them if they are paying the correct attention.
If your belayer doesn't do this consider talking to them (ideally when you get down "next time can you not do that, etc"), they may not see it as an issue, but if you do they should adjust their technique according to what you want (climber is always in charge).