Compasses are good equipment both spelunking and diving. Even the deepest cave you could go to is still near the earth's surface, geologically speaking. The earth's magnetic field is also essentially the same under water as above.
If you are using a compass, what you need to be aware of is nearby magnets and large sources of iron. So if you were exploring an underwater metal shipwreck, it could throw off a compass, the same as if you are testing a compass in your house standing next to a metal stove or refrigerator. The same is probably true if you explored a cave with a very concentrated source of iron ore. If you store your compass with a strong magnet, you risk demagnetizing it so it won't work at all. But if you treat it properly, a good magnetic compass is going to be more reliable than any electronic ones I've tried.