A few years back at my University climbing gym, someone posted a study performed on chalking your hands while climbing. Basically, it was discovered that there was actually a measurable decrease in the coefficient of friction when you used chalk. As in, chalk made climbing holds harder to hang onto.
This of course made all the climbers who read the abstract laugh, because every climber knows that chalk helps you hang on to slippery holds. But after reading the study I was actually convinced. Chalk will soak up all your sweat and grime and dry your hands, but clean dry hands are actually gripper than chalked hands. In fact, according to this study, even wet clean hands have a stronger coefficient of friction than chalked hands. The conclusion of the study was that climbers would be better off using plain water instead of chalk.
It's the oils produced by your skin that makes rock holds slippery, and water naturally displaces oil, whereas chalking you hands puts a fine powder between you and the hold.
Whether or not you have chalk, the science says you're better off washing your hand in water. I've yet to try it (mostly because the rock I climb is so dirty I don't even need chalk) but you could try climbing with a wet sponge in your chalk bag and a golf towel hanging off your harness to wash and dry your hands with when they get grimy.