Both Icelanders and tourists love soaking in hot pools outdoors, especially on grey winter days. The tradition dates back to the Viking times. You can combine it with the hiking trips, ending the day with dipping yourself in the thermal waters, surrounded by wintery landscapes.
One of the most popular 'hot spots' is Blue Lagoon with bright-blue water surrounded by lava fields. Moreover, it's quite close to Reykjavik. You can find hot pools all over the country, some of them with more infrastructure, while others being simply hot springs in the nature without any facilities. When you head to Iceland in winter, make sure to bring your towel!