I'm watching videos on how to lower while outdoor climbing (sample video) and it seems that the accepted practice is to always be attached to the wall with both a rope and the PAS. But what is the rationale for this? Is it expected that the PAS is less reliable than the rope? Is it because the rope might slip out of your hand as you're untying it from your harness? Obviously the holds themselves could fail but that's an equal risk regardless of your method.
In general, redundancy is key. Second, it is critical to retain the rope at the top of the climb. If the rope falls with the climber anchored to the top, you're going to be in for one heck of a time. If you only have the two of you, you're likely to require outside assistance/rescue. Therefore, the rope should stay physically attached to the climber in some way at all times.
One thing you might not know, depending on how much climbing you've done, is that rope is heavy. I've been at the top of a 100-foot pitch, and let me tell you: you can really feel the weight of the rope. The weight alone can be 4-5 pounds, and that's not counting any friction. Trust me, you will really notice 4-5 pounds pulling on a rope if you're trying to manipulate it. It's far too easy for it to slip out of your hand as you're threading the end through the rings. When it's secured to you, that's not even a worry at all.
As a side note, when "cleaning" a route, you're not really holding onto any holds. Both hands are active, and you basically are suspended via the anchors, with your feet just providing some stability. If by "Obviously the holds themselves could fail but that's an equal risk regardless of your method." you mean the anchors themselves, then yes. Otherwise, there aren't really any holds that you're actively using.
Breaking down the video a bit further:
In the first method shown, the climber attaches themselves directly to the anchor, pulls up slack through the rappel rings, then ties themselves in before untying their original double figure eight knot. Realistically, they are only properly protected by the anchor as they manipulate the rope, especially as some methods I've seen have the climber "off belay" as they work at the top. Even if the belayer is prepared to catch you, you're going to have a bad time if the anchor drops you while you're fiddling around up there.
In the second method, the climber follows a similar method, only they tie a figure eight on a bight and attach it to themselves, before threading the end of the rope through the anchors (not the best plan, especially if they aren't exactly the right type of bolt). Honestly, I personally wouldn't trust a single non-locking carabiner with my life when it's not already loaded, but some people do, it seems.
My personal preference includes using 2 quickdraws at the top (opposite and opposed) before messing around with attaching myself to the anchors. That way, you are much less likely for a slip while messing around to unclip your rope from the carabiner there.
TL;DR: Never have the rope in a position where it would slide out and fall to the ground without your active intervention. It must either have a stopper knot (when preparing to rappel down) or be tied to your harness in some way. In some systems, you don't even properly "tie in", but simply attach the rope to your harness (via an overhand on a bight, say) as you untie your main knot. Your sling/anchor system can be fully trusted to hold you (though I like attaching to both anchors if possible) but you can not let the rope fall down. Without it, you may be stuck, hooked into the anchors, for a while as your friends/the authorities try to rescue you.