It's always OK to move fallen debris from the trail, assuming you are sure you are on a real official trail. Make sure you don't accidentally remove "brushed in" trail entrances. That is where brush was deliberately piled so that a trail is not used, hopefully eventually reverting to just woods again. If you're on anything with clearly deliberate blazes, you're safe.

For anything more than clearing debris without tools, you need to ask the owner or administrating agency. Public agencies usually have rules that they don't want any unauthorized trail work. This is just to make sure they don't get morons doing stupid things and to avoid liability. If you get to know the manager, forester, committee, or whatever that oversees a particularly area and they get to know and trust you, you usually end up with a informal arrangement where they trust you doing trail work without asking first and without them formally giving permission.

Some groups can be real control freaks and don't want anyone, even those that clearly know what they are doing, doing trail work without 3 levels of approval. I know one organization that takes this to the extreme. They have a "steward" assigned to each property, and then a few master stewards for groups of properties. Even though I actually built some of their trails, I can't go out there when I have a spare hour and clean up a trail without prior approval. I don't feel I should work that hard to give away my time for free, so I find a different place to do trail work. There are plenty of places run by other groups or organizations that are happy to have me help.

I'm on the trails committee in my town, so I am actually authorized to go out with a chain saw and clean up a mess on any of the town land whenever I see a need. However, we don't want people going so far as using a chain saw without talking to us first, but even then there are certain individuals and groups we know aren't going to make a mess and nobody is going to make a fuss about one of them going out and clearing a major blow-down. But, we're not going to tell you that if you come and ask as a individual, especially if we don't know you.

A good way to get known is to volunteer for organized trail work. I don't know how other towns work but we probably have 1-3 formally organized events a month except when the snow it too deep to do much useful. After you show up a few times, the organizers will recognize you and you'll be trusted a lot more. You might even mention a particular trail you think needs some work and offer to take care of it over the next week or two. Once you're known, you'll probably get a quiet unofficial nod and it works out for everyone.