I wish there was a solution that's both practical and 100% data secure, but I don't know of one. An open source phone, as suggested by Matthew, is probably the closest, but with that you unfortunately still need to expect to run into reliability problems – not something I would risk in the outdoors.
Also, open source phones are probably among the products that intelligence agencies and police are most interested about because of their potential use by terrorists and criminals. And as much as I love free/libre software, unfortunately it is not immune against backdoors. So these phones are more likely to leak data to police etc., than a device that doesn't have communication hardware in the first place (which also makes it less useful to criminals).
Here's a solution that I found secure enough for my personal paranoia, and is still very practical:
Get a cycling computer
I used to use a handheld Garmin etrex10 – which, as was already mentioned in the accepted answer, is pretty sure to be safe with respects tracking. Maybe a good idea to get your hands on one of these while you still can.
But it needs to be said, in practice these old bricks are less than ideal. Always manually uploading the maps for the (pretty small) regions that fit in its memory is tedious indeed. I generally travel by bike, and then you quickly run into the situation that a rectangular map with everything you need doesn't fit in memory, so you need to pre-crop around the route you want to take – it's a pain. And at some point I realised that my prior online research for this probably gives away more data then a normally-used GPS would... (Yeah, you could VPN/Tor etc., but a) will you? and b) online will always be more risky than offline.)
This kind of device is probably best used with actual physical maps – which is arguably still a good idea anyway. I personally can't really be bothered...
When my etrex10 eventually broke down, I got a Garmin Edge 830, one of the better cycling-oriented ones. Now, this is definitely not an answer to “no transmission capability?”, because it does have BlueTooth.
Still, I'm pretty confident that the way I use it does not give away any information. Why?
Well, these “cycling computers” are nowadays designed with the use case in mind of having them coupled to a phone, as most cyclists will always have a phone with them as well. Alternatively, they will connect via USB to the Garmin Express application on a PC. Both of this definitely can leak data. Whether this is in fact exploited by any authorities, I don't know – but it's certainly a possibility.
However, because of this standard user case, the Edge 830 (and presumably many other cycling computers) really don't have any WiFi† or cellular network hardware, so you can be pretty sure that they won't do any long-distance communications behind your back. So the only threat you might still be worried about is that the phones of people passing by might pick up the Bluetooth signal.
The advantage of a modern device like the Edge 830 is that it actually has a large built-in storage, and comes with a complete and up-to-date OSM and topographic data. That does a lot against the temptation to consult Google Earth / -maps etc. first. Really, the Edge 830 fulfills most of my planning needs in addition to the navigation ones – without ever needing to go online.
Of course, since it's designed for cycling, the routing isn't always best for hiking – but in my experience the MTB mode actually works very well for hiking too, since it makes use of even the smallest trails that OSM knows about.
You can always also upload GPX files, to follow whatever routes you want. That does require connecting via USB, but I only ever do that on a Linux machine without any proprietary Garmin software on it. The Edge 830 is then simply mounted as a USB mass storage device, and only data you explicitly want to transmit is exchanged.
†Somewhat confusingly, the Edge 830 does advertise itself as WiFi-capable, but it turns out this is only usable via a phone or PC. Theoretically, it could of course be that it does secretly do WiFi or cellular on its own... but then, the same could be said about your flashlight.