I don't know about poison oak, but have lots of experience with posion ivy. I think the irritant in both is similar (urushiol oil, or something like that). This answer applies to poison ivy, which I think transfer to posion oak too.
No, ordinary soap does no good against this oil. I have taken a shower with ordinary soap shortly after being exposed to poison ivy, and got the nasty rash. Last year I started using a product called TechNu (or something similar to that, can't remember exact name) and I can tell you it does work. It is something you use like soap after exposure, but unlike soap, it actually does wash off the irritating oil.
There have been a least two cases now where I'm quite sure I got exposed to poison ivy while doing trail work. I went home afterwards, took a shower and washed with the TechNu stuff, and had no symptoms. I'm quite allergic to poison ivy, and am convinced that washing with this stuff made a difference.
I'm at work now and the bottle is at home, but I sortof remember that poison oak is listed on the label as one of the irritants it is effective against.