What material – such as polyester, polypropylene or wool – or combination thereof, both works to keep you warm when wet and dries quickly, as an alternative to neoprene? I'm looking for something cheaper and simpler than a dry suit but less restrictive than a wet suit, for paddling in cool but not very cold water. Something you can submerge in and still keep reasonably warm.
I would say that it would depend more on the air temperature.
If you have cold water and rather warm temperatures, such as rafting on a cold river on a hot summer day, then synthetics like polyester or polypropylene would be the way to go since they will dry out quickly and not retain the water.
There are also Polyolefin swimwear which are designed to provide insulation and warmth.
Where wool has an advantage is for fisherman/sailers who are spending time in much colder temperatures, since even if wool doesn't dry quickly, it retains its insulating properties even when wet. It looks like it was used for swimming suits but hasn't been used since the 1930s in the US.
Fleece dries fast, and fleece is relatively warm while wet. It's probably the best you're going to get in normal clothing materials. But it's not an alternative to a wetsuit or drysuit, more of a "I'm going hiking with a light jacket in autumn" option, or an addition for below a wetsuit. If that's what you're looking for in terms of how wet you'll get, how long you'll stay wet and how cold the weather will be, it's a good option. Add a dry top/anorak to keep the wind out, and you've got a decent flatwater kayak/canoe combination.
Thermal shirts (thanks for the correction Martin) sometimes called thermal underwear, often made from some combination of polyester and viscose or such, are a good option as well. They're a lot thinner than a good fleece sweater, but are form fitting and surprisingly warm for their thickness. A thermoshirt and an anorak works maybe just a bit worse than fleece and anorak, a thermoshirt alone can be nice of a warmer day if the water's not too cold. They can also be layered.
When doing even light whitewater, sea paddling or longer trips think about it a bit, try to build up some experience so you'll know your limits and those of your gear. Better safe than hypothermic.
These are technically a baselayer for wearing under a wetsuit, but are often worn during the warmer months as the only layer for watersports. They're less restrictive, dry reasonably fast but designed to keep you warm when wet. Available in various weights from lightweight non-UV resistant to heavyweight thermals. Watch out for that UV resistance on cheap products.
Cost wise the cheapest would come in at slightly below the cheapest wetsuits, but expensive gear should only be about 3-5 times that base price.
This is probably the best option given your description.
Again available as watersports specific, these are a step up from a rashie. More expensive but warmer, often valid as sole equipment down to quite low temperatures and liable to cause overheating if it's not cold enough.
This is a fairly expensive option. It acts initially as a waterproof layer but once water gets inside it behaves much as neoprene does. Aquatherm gives you more freedom to move than wetsuits due to being a much thinner material. Competitive kayakers swear by this stuff for winter training. Expensive but cheaper than a high quality wetsuit.