I swim decently - no pro but I can do a couple of km, resting once in awhile. Many times, both in day and night, in the warm, summer, Mediterranean sea, friends have cautioned me to "not swim so far out".
This has happened both in day and at night; I suppose there's a different answer for each. The water never gets cold at the distances I go, there's no jellyfish or anything like that out there, and I watch out for boats.
So, what's wrong with swimming out far?
There don't seem to be any detailed statistics, but common-sense will tell you that open-water swimming away from the shore is not risk-free. The hazards can be split into four categories:
1 - Wind and currents
Even in the Mediterranean (Med), wind and currents can be problematic. On one windy day in 2013, for example, seven swimmers drowned along just one stretch of the French coast. The weather can be unpredictable and sudden storms are common. The farther out you are, the more wind and current will come into play.
2 - Boats
You may keep an eye open for boats - but are they keeping an eye open for you? A lot of people in hire boats barely know what they are doing, and every year bathers are run down. As a swimmer you are not very visible, particularly at night.
You can reduce the risk a little by wearing a fluorescent cap and trailing a high-viz float:
The farther out you are, the more you are at risk from boats that won't be on the lookout for swimmers.
3 - Sea life
Jellyfish in the Med are a significant danger, with over 150,000 swimmers being treated each year. Stings can be toxic enough to cause you difficulties. I'm not sure how you can be so confident that there are no jellyfish where you swim.
The farther out you are, the more difficult it becomes to get back to shore if you are badly stung.
4 - Sudden illness or incapacity
Any sudden incapacity in deep water can be fatal, so cramp, heart-attack, stroke, asthma etc will have more serious consequences, particularly if you are swimming alone.
A friend of mine drowned while swimming in warm water when it seems that he simply fainted. He was young, fit and healthy and had never fainted before, so we'll never know why it happened. A prominent member of the wild swimming community in Devon where I live was found drowned while swimming solo in open water. He was super-fit and highly experienced.
You can improve your chances of surviving an incapacity by learning the Drownproofing technique, which enables you to float efficiently without full use of your arms and legs.
Clearly, the farther out you are the more difficult it becomes to survive any sudden incapacity.
Conclusion
So while you couldn't say that offshore swimming in the Med is unjustifiable, the dangers are significant.
Especially if you are swimming alone, it would be more prudent to swim parallel to the shore where you can reach safety quickly if you get into trouble.
Is it really worth the extra danger to swim farther out? Only you can decide - but do it with a realistic understanding of the risks.
The other answers already told you about exhaustion, I can tell you firsthand how dangerous cramps could be.
I was swimming together with my former girlfriend in Ustka, Poland on the coast of the Baltic Sea. I would say that I am an average swimmer with a good constituition so I can swim several kilometers and I never had cramps before.
There was no warning when suddenly my right leg was strongly cramping. I was thrown out of balance by the unexpected shift and swallowed some water.
Now there are two things about Ustka: The beach has an extremely gentle slope so you can walk out hundred meter and you are still standing. The other is that the water is brackish water, it does not sting and it causes no coughing. Luckily I was still in standing depth, 1.7 m and therefore it was not a problem. The sea was also calm.
Being curious by nature and being safe, I tried out to stay afloat with the cramping leg and simulated that my other leg could not move either. Yes, you can stay afloat, but it took me several minutes to master it. Essentially I must submerge my body almost completely, breathe deeply and put my body in a relaxing position. If you are able to dive, you could pull your toes to your body to loose the cramps.
So I could stay afloat, but what now?
Every time I tried to raise my arm (I did not raise it fully because I did not want to alert any guards), my body was submerged and I could not breathe anymore. It puts me out of the relaxing position and every time it was hard to get again in the floating position. The same problem when I forcefully blew out air to simulate screaming, I sank and I often lost the necessary coordination to stay afloat
I could not move. For someone who does not experience it, it is very hard to imagine how strongly the movement is inhibited. You need the legs to balance yourself, without them you cannot execute effective swimming movements anymore.
So essentially I cannot scream, I cannot alarm people and I cannot move. The whole thing was problematic enough with one leg cramping, I was feeling like a turtle on the back, if the whole thing would be realistic, I would have drown. And I would say that if the leg really cramps the first time, you are not able to move correctly to stay afloat.
ADDITION: @Headcrab pointed exactly out what I want to warn against: You are deluding yourselves if you think it is the same if you are in the swimming pool and simulate cramps by holding the legs stretched. Sure, that is also easy for me: Simply blow yourself up as much as possible and scream and move your arms as you like. It does not work this way.
It is very hard to describe what exactly is happening, but I do my best to explain it: Even if you do not move, your brain is in control of your extremities and can anticipate the correct movements to float. So it can balance out problems almost subconsciously. Once your leg cramps, your brain cannot anticipate your leg movement nor can it correct it, so you lose your balance almost immediately (which resulted that I swallowed water). In this case your brain almost immediately switches to "RED ALERT" and you are prone to panic which is not a good thing. After finding ground immediately (I knew that, but your brain needs reassurance), the panic ceased, but the lingering impression how fast you can lose control is still on my mind.
The cramping leg caused that I need to use my torso muscles to hold the leg up (you need to lie on your back to put your mouth and nose on top) and in an unusual position; I was lying lower in the water than normally. Also I needed to regulate my position actively and clumsily to hold my balance and I really needed time to allow my brain to adjust to the new conditions; if I make too strong moves, I lose my balance. As my left leg was only "simulating", the water was calm and not irritating, the real condition with two legs crampings could be only much worse.
As muscle tissue is denser as water and I was really almost submerged, I am inclined to believe that a more muscular man than me cannot simply hold himself up. Having different body types, other people may experience it different, but for me neither screaming nor waving was possible without submerging again.
There are two sources of danger, both of which are unexpected.
The first has to do with the variations of your condition. Based on what you have written, you can do say, two kilometers on an "average" day. Then it seems that you can swim safely one kilometer out and one kilometer back. But there will be some days that are not so average, one when you can swim only 1.5 kilometers. If you swim 1 kilometer out on such a day and head back, you may end up 0.5 kilometer short and drown. This is particularly true if you get a cramp and there's no one around to help you.
The second has to do with variations in the sea's condition. Again, there will be no troubling winds, tides, currents, boats, jellyfish on an "average" day. But there will be one day when there will be a problem of this sort that you won't be able to handle, especially if it catches you unawares.
The above events could be 1 in 100 (or even 1 in a thousand). But if you go out 20 times a year, you can expect to encounter a 1 in a 100 event in 5 years (50 years for 1 in 1000).
Basically, you should stay close enough to the shore to be 1) near other people, and 2) to be on or near water shallow enough to stand (or push off a shallow bottom) if necessary.
One reason is geometry.
If people on the shore know the approximate direction you went to swim, your possible location could be anywhere inside a circular sector. The area of this sector depends on the square of the radius:
Each time you double the distance to the shore, you divide by 4 the chances of someone finding you if you ever need help and cannot be seen or heard (e.g. due to waves or obscurity).
It gets worse if you begin to drown. Assuming a constant slope of the sea bed, the volume of water that needs to be searched depends on the cube of the distance to the shore.
Others have already pointed out the dangers of changing weather conditions, tides, sealife and other people (boats, jetskis, etc) but I think another danger is heat exhaustion, heat stroke and heat cramps. All of these things are caused by exposure to heat, lack of hydration and other factors. If you're swimming for a while and you haven't been careful about your fluid intake or sun exposure, you could get into trouble.
Other answers only mentioned that tides can be dangerous, but didn't go into details. And I am not sure how applicable that can be to the Mediterranean sea.
But tides can be very dangerous, when the sea drags away from the beach. This once happened when I swam in the Atlantic ocean. Fortunately, I didn't go far away, and noticed what is going on. I was swimming like crazy and managed to make it to the land.
More details about this in "Rip currents: The ocean's deadliest trick".
A lot of these answers have talked about the specific risks involved in swimming. Here's a different angle: the danger of any activity is related to a) how risky it is (the other answers) and b) what you can do about them - ie. what is your escape plan if something goes wrong?
Swimming deep in the ocean, you're basically relying on your ability to keep swimming, or failing that, remaining afloat until you can be rescued or drift back to shore. As far as "Plan B's" go, both of those suck!
So, short answer - it's dangerous because if anything goes wrong, there's not much you can do about it.
Not keeping a eye on the tides. Near shore or between islands the tide going out can sweep you out to sea. Once back on the beach it can be a long walk back to your flipflops On hot sand rocks coral & such. Just because you fail to see the shark does not mean he his not there. Other beach goers do not pay attention to you way out there. It is a long swim back from out there with fish & speargun. I would advise if going out that far to take a inflatable object with you.
Another reason which I think is not mentioned before is something runners or other long duration sports people experience which is called the 'man with the hammer'.
If you would swim for multiple hours (even with some short rests), the body is using its energy reserve. When that is finished, the fat reserve is used. However, the conversion from fat to energy is very inefficient.
When this happens a runner cannot 'run' but only walk. Now if you are swimming, you cannot use your full energy but only a small portion of your energy. This is in principle not a problem, but if you than get into other dangers (like current, a boat coming towards you etc), you CANNOT swim fast(er) anymore.
In most area of the Mediterranean currents are not a big issue and tides are negligible. Unless there is a big ship passing nearby at high speed the sea won't get rough all of a sudden, so if you have a cramp or suddenly feel exhausted you can just rest for a while floating. These are minor issues.
However the Mediterranean is a crowded place and everywhere you'll find a lot of speedboats. Security enforcement is poor and a lot of people at the helm are reckless, even if you see them they still might constitute a danger.
