The article Iron Production in the Viking Age on Hurstwic discusses sources of iron available in the wilderness of Scandinavia and Iceland. To my knowledge and experience, similar sources exist in what is now eastern Canada. From that article and other reading I identify two primary sources of iron people gathered from the wild for primitive tool making:
Bog Iron
At the Norse settlement in L'Anse aux Meadows in Newfoundland, Canada, there is clear evidence that Norsemen harvested and smelted bog iron to use as the raw material for the iron rivets they fabricated to repair their ships there 1000 years ago.
Where streams run from nearby mountains through a peat bog, bog iron can almost always be found. That combination was probably visible to Norse explorers even from on board their ships in the cove at L'Anse aux Meadows. It occurs in glaciated regions throughout the world, and so would have been very familiar to the Norse explorers at Vínland.
Streams carry dissolved iron from nearby mountains. In the bog, the iron is concentrated by two processes. The bog environment is acidic, with a low concentration of dissolved oxygen. In the acidic environment of the bog, a chemical reaction forms insoluble iron compounds which precipitate out. But more importantly, anaerobic bacteria (Gallionella and Leptothrix) growing under the surface of the bog concentrate the iron as part of their life processes. Their presence can be detected on the surface by the iridescent oily film they leave on the water, another sure sign of bog iron. In Iceland, the film is called jarnbrák (iron slick).
When a layer of peat in the bog is cut and pulled back using turf knives, pea sized nodules of bog iron can be found and harvested. Although the iron nodules are reasonably pure, there aren't many of them. They are, however, a renewable resource. About once each generation, the same bog can be re-harvested.
Iron Ore
In some regions (particularly Sweden), iron ore, rather than bog iron, was the raw material for smelting.
Iron ore is fairly abundant in many parts of the world. According to the Wikipedia page on Iron Ore, the color of ores can vary.
Iron ores are rocks and minerals from which metallic iron can be economically extracted. The ores are usually rich in iron oxides and vary in color from dark grey, bright yellow, or deep purple to rusty red.
The concentration of iron in ore also varies, and for iron ore to be practical in primitive tool construction, it needs to have a certain amount of iron present. That same Wikipedia article notes:
Ores containing very high quantities of hematite or magnetite (greater than about 60% iron) are known as "natural ore" or "direct shipping ore", meaning they can be fed directly into iron-making blast furnaces.
