You certainly can, if the waist belt isn't high enough and tight enough.
A few years ago I briefly worked at a small rock wall where among other duties I belayed and showed people how to put on a harness. In my experience most newcomers don't (want to) fit the waist belt properly, and a significant fraction of regular climbers fail here too.
The waist belt needs to sit above the hips, not around the hips, and certainly not falling off the butt as some find fashionable with pants. It is not sufficient to casually tug at your harness and say "look it doesn't slide down." In a fall forces many times body-weight can occur: the padding will compress, the textiles will stretch, and your body will deform. The harness needs to hold on the pelvis not on the buttocks or other fleshy bits.
Look at images of the human skeleton: the thick part of the pelvis is at the top, the iliac crest. That is where the belt should hold. If it slips (or starts) below that point only the projections of the femurs exist to catch it, and they are mobile and of insufficient dimension for the task.