Yes, you can build a basic thermometer using just water, or any liquid. You first need to mostly seal it in a container (like a glass bottle) and use a straw or thin clear tube immersed in it to allow the liquid to expand up the straw. Do not let the straw touch the bottom. A 50-50 water & alcohol solution will also do the trick if you happen to have alcohol, but in a survival role alcohol should be kept for injuries, etc.
The trick in order to re-use the thermo-bottle it is to make sure that each day the exact same amount of water/alcohol is there at the start since both water and alcohol slowly do evaporate over time. (Note your regular mercury thermometer is sealed so it never escapes.) Or seal it all when not using it.
The next stage is marking the clear tube with various temperatures. Best case scenario is if you are in winter and can place it in ice cold water. That's your 0°C/32°F mark on the straw, usually the bottom of the straw. Placing the thermo-bottle in boiling water is an upper mark on the straw, just be careful not to burn yourself, or crack the bottle, or even melt the bottle or straw. Placing the thermo-bottle under your jacket for 10 minutes should give you a 37.0°C/98.6°F reading or body temperature (or you can immerse the bottle in your fresh urine which should be about 37°C/98°F).
Once you have these various marks, divide the straw into equal sections to give you an approximation of the temperature between your body temp and zero. It is still an approximation because altitude, humidity, pressure and water density can affect the reading. With alcohol as a liquid, readings below zero will also be possible.
For efficiency, practice using and marking a practice thermo-bottle and straw or tube BEFORE going into the outdoors when you do have a thermometer to help. This will help you get the kinks out when you finally need to construct one.
Other ways to measure temperature without a proper thermometer are:
A Galileo thermometer (more complicated tube with water and floating weighted orbs in it)
Evaporation rates of a water drop in direct sunshine (done ahead of time and memorized.)
Facial sensation (the face is better than hand at discerning temperature)
A basic personal chart (ice freezes at 0°C/32°F, my breath is visible at 7°C/45°F, I go to t-shirt mode at 16°C/61°F, I start to sweat when sitting at 28°C/82°F, my metal camping spoon in 43°C/110°F sun for 1 minute scalds me)
Here's a related site about the thermo-bottle (not my site):
https://www.stevespanglerscience.com/lab/experiments/water-thermometer-sick-science/
Good luck.
PS do not throw boiling water in freezing temperatures, it's not worth the risk.