This map (and the Wikipedia article) will explain why you don't have a problem (in the UK, I'm guessing), but others do, elsewhere.
In the UK, you can ignore it at the moment, but you need to understand WHY you're ignoring it :)
As an aside, compare this with the declination in 1872... Then, the declination error in the UK would have been between 20 and 30 degrees west!
Typically when walking on a bearing you need to allow a margin of error of about 5°. (ie, when 'aiming off', allow at least 5° of error). So the sub-one degree at present in (most of) the UK isn't a big issue.
One thing other haven't mentioned is the precise impact of ignoring the declination. Let's imagine we need to pace 500 metres on a bearing in thick Welsh fog across typical Welsh moorland to reach a bridge on a stream. We know we can't possibly hit the bridge directly, so we need to 'aim off' to a point upstream, and then walk downstream to find the bridge.
You can only follow a compass bearing accurately to within +/-5° (errors both in working the bearing out on the map, and actually following it on the ground. A 5° error over 500 metres would mean we miss our target by about 44 metres (500 * tan(5 degrees)), so we'd want to 'aim off' by that amount as a minimum. Let say we aim off by 50m.
Now, if the declination is 1° and we ignore it, our worst case error angle changes to 6°, meaning we could end up 52m from our aim point - very slightly downstream of the bridge.
But if we ignored a declination of 10°, we'd be up to 134m from our aiming point, and so far from the bridge we'd miss it completely.
Of course, there's one thing worse than ignoring declination: That's adding it when you should be subtracting, or vice-versa. The potential 25° error (10° declination, wrongly applied, plus 5° general error) would put you almost 250m from you intended destination.