It also seems the water dispenser stopped working. But the two did not stop working at the same time. Can I use a heated blow dryer near the ice dispenser to thaw it out so that it will work again?

Your are assuming that the problem is frozen water lines. That may be but it may not be. You should determine what the problem is before you try to fix it. It could be a blocked water line, it could be defective solenoid fill valves or it could be some other problem like frozen lines.

First trace the water supply tubing back to where it connects to the house piping. That will probably be under the kitchen sink. There you should find small valve with a "T" handle. These valves make a very small hole in the pipe which can become blocked. Close the valve and then re-open it. If it is blocked that will re-open it.

Next trace the water supply tubing to the solenoid fill valves for the ice maker and the chilled water on the refrigerator. These be in the back or under the bottom in the front. There may be two separate valves or may be two valves built into one. The water supply tubing will come into one side and there will be two lines out the other side. One will be for the ice maker the other will be for the chilled water. Disconnect both output lines. Press the handle for chilled water. Water should come out one of the valves. If it does, unplug the electrical wire on that valve and put it on the other valve if possible. Then press the chill water level again. Water should come out the other valve. That will confirm that the valves are working. Put the wires back on the proper valves and reconnect the water lines.

If you do not get water out of the valves, turn off the valve under the sink. Disconnect the water line into the solenoid valves. Turn on the valve under the sink briefly to see if you get water out the input water line. That will confirm that water is getting to the valves.

After doing all this you can begin to consider that the lines may be frozen.

PS
After thought, if your water line has a filter check that first.