This is highly dependent on the type of water as well as the location, but I'll summarise a few things to be aware of. In many locations most of these dangers won't factor in, but they're useful things to bear in mind if you're trying to assess the danger of a particular body of water. I'll focus on the sea here - for other things such as rivers similar things apply (but there's obviously other things to look out for, different kinds of wildlife, waterfalls, etc.)
Currents: Yes, there are areas of sea that can "pull you in". (The same obviously applies for rivers here, but this is less commonly thought of with the sea.) More dangerously, there are also currents that can "pull you under". Similarly, and sometimes just as dangerously, there are areas of the sea that can "pull you along" the coast horizontally. This can sometimes be dangerous simply because it's much easier to miss, and can pull you into dangerous areas such as:
River outflows: Along the same guise as currents, but these can often hide at full tide and seem to appear very suddenly and strongly if you move into them.
Quicksand: Not often thought of when you're in the water, but if you decide to stay on the bottom for a while and hit a patch of this, it can root you to the spot.
Watercraft: Again, something that's often missed, but stray jetskis (for example) can seriously injure or kill you if they hit you full on.
Hidden depth: While you can often expect beaches to fall down at a gentle angle, there are often hidden areas that can suddenly drop away. While not an inherent problem if you're a good swimmer, these can often be associated with undercurrents too - so beware. Personally I'll only swim on beaches I'm familiar with at high tide (and have seen at low tide) to try and mitigate this somewhat.
Wildlife: This varies greatly - some areas can have deadly sharks (or similar) in them, and these risks go without saying! However, bear in mind more common things such as jellyfish (some of which will just give you a nasty sting and others that can be lethal) and crabs (which will give you a nasty nip if feeling threatened.) Know what lives in the waters you're swimming in, and be prepared accordingly.
Sharp rocks: This goes hand in hand with wildlife in that sharp limpets can also live on rocks, making matters worse - but just because the entrance into the water looks sandy, doesn't mean it is all the way through. If you swim at high tide and stay in while the sea recedes, you may find yourself painfully clamouring over all types of nasties. Same goes if an unexpected large wave throws you against such rocks.
Pollution: Like any body of water, you should be aware of pollution - especially after storms many runoffs will quickly head into the sea containing a variety of toxins and bacteria. In some countries / areas factories are still permitted to dump waste into the sea directly, in this case I wouldn't even risk getting in the water!