(TL:DR -- The "safe" amount to harvest varies enormously with plant species and context. When in doubt, take 5% or less. 1/3 is probably safe for common, prolific species.)
This rule is given out because if you harvest every fruit on a plant, you've stopped it from reproducing this year. You don't necessarily even have to harvest all the fruit, if the plant has a generally low seed-germination rate. It gets a little more complicated if you're harvesting things other than fruit, too: taking a wild carrot kills it outright, meaning that unless its seeds have already ripened and been dispersed, that's 100% of its reproductive capacity gone. Same, and even worse, for annual species harvested that way.
In scientific collections, the general rule I was taught is to leave at least 95% of a population -- that is, to kill no more than 1 in 20 plants, or to strip no more than 5% of the population's reproductive capacity. This is a good solid rule, especially when you're dealing with relatively uncommon wild species or species that are slow to recover from harvest. Frankly, though, foragers shouldn't take any of those unless it's literally a matter of starvation.
Elderberry* is neither rare nor slow to recover; it flowers and fruits prolifically, year after year. You could likely harvest a third of its fruit, or even more, without damaging the population's long-term reproductive capacity. Be careful to leave enough for the animals that depend on the fruits, though! Personally, I definitely won't take more than 1/3 of a native species population.
All that said, there are some edible plants whose extirpation would be a public service. Look up your local noxious weeds, and harvest those to your heart's content. Take more than you need! Take all you can carry! Kill them off, please!
Did you have any specific plants in mind, other than elderberry? I could do some research and get you firmer numbers for them.
* Assuming you mean one of the more common Sambucus species -- S. nigra or S. racemosa or something -- in North America or Europe. I'm not well-up on the whole genus.