In all likelihood, you just need to replace itthe helmet. Nearly all helmets, climbing and bicycle helmets included, need to be replaced once they become worn or older than 5 years. Ideally, you would track how old eachthe age of your helmets areeach helmet. If you don't know how old one is and it looks well used, I wouldn't risk it.
Stick to gently wiping it with a soft dry cloth. Trying to clean a helmet with water or most any kind of soap only risks damaging the foam. If a helmet is so worn that a dry cloth doesn't restore it, it might be too warnworn. Consider how important your head is before try to cover it with compromised foam ( or compromised plastic, etc ).