I want to buy a canteen, and I have choice between aluminum, stainless steel, enamel and various kinds of plastic. I had canteens before that made any liquid taste horrible after just few hours. I would like to find one that does not affect the taste of water too much.
If you wash the canteen thoroughly before using it then it shouldn't affect the taste too much regardless of what material its made from - I'd recommend this anyway just as a matter of course. Wash once, soak overnight in hot, soapy water then wash again.
Where this might come into play more is your lips physically touching the metal when you're drinking from it, which may well make it taste metallic - I've found this as well with several metal containers and tend to prefer plastic ones for this reason.
Run the canteen through a dishwasher a time or two and it will get rid of the odd taste.
Sometimes the bad taste does not come from the bottle itself, but from the lid. I had an old aluminum canteen that gave a strong rotten taste to water, which came from a cork seal that was a part of the screw on lid. Replacing the cork seal with a new one fixed the problem.