Consider the question in the terms of basic physics. Essentially, the magnetic pull of the earth and needle's magnet creates a torque on the compass (wheel / disk / needle etc. ) causing it to rotate. In an ideal compass, the torque meets only dampening resistance - you don't want the needle to move too quickly or you will get an oscillation while hunting for steady state. Thus, the needle will move until it has zero torque placed on it (pointing north)
In a non-ideal compass (a real one), there is some amount of force that counters the rotation of the compass (generally friction). The greater that force, the more potential error that a compass may have. This is true because the force of the earths magnetic pull produced the greatest torque when the needle is at 90 degrees to it. As the needle gets close to pointing north, the exerted torque from the earth reduces. Eventually, the torque exerted by the earth is not enough to overcome the frictional force (s) and the needle no longer moves (even though it is not pointing north). Thus, your reading will be in error.
Cheaper compasses will have a greater frictional component. Often, this component is a function of the angle at which it is held - making the inaccuracies harder to predict.
A good compass will be manufactured with emphasis on reducing frictional forces and will often have a level to help you keep the compass flat while performing readings.
I can't speak to what the price point difference is between good and bad. I can say, though, that a compass has a very specific purpose - to help keep you from getting lost. Do you really want your survival to depend on the lowest bidder?