Obviously cross/back-country skis can be damaged by skiing over things like rocks, stones, gravel and asphalt. What about other, softer objects exposed when the snow starts to melt, like sticks, roots, leaves, fir tree needles, soil, etc? How damaging to the skis can these be to slide over?
You definitely need to be concerned if you are using waxless XC skis with fish scales under foot. Skiing over hard dead sticks can break off the edges of these scales (or wear them down over time) making them less effective on climbs.
For smooth bottom skis, pine needles, roots, etc will likely do nothing more than scrape the wax off your skis (which if you are using XC skis with kick-wax, you might need to re-apply mid-day, while if you are downhill back-country skiing, having a little glide wax in your bag might not hurt.) See this answer about how often one should wax skis.
Other than that, you do run the risk of more serious damage (gouges) by skiing over hard dead branches or roots, so be gentle. At the end of the day, if there is visible damage to your skis, repair and re-wax as normal.
Otherwise, if there is no physical damage, it might not be a bad idea to re-wax anyway given that you know you skied over somewhat abrasive surfaces.