I've really started to notice how cold it is when near our windows when the temp is below zero.
You can see the caulking between the windows is dried up and cracking..
Is the seals gone?
How can I make it warmer around the window areas?
Thanks if you can help me!

By double pane do you mean thermopane windows?

Caulking should be fixed. This will not help warm up sides of windows, but will protect from water intrusion.

If the gasket between window panes is cracking, then thermo seal probably bad. Would have to replace glass units.

If these are replacement windows, when they were done installer did not insulate the void left between window and studding. U might be able to insulate there by removing trim casing.

Chuck

I have installed all makes and kinds of windows for over 25 years. All windows are cold at zero even with proper insulation. Glass has the worst R value of any product.
If the caulk is cracked, the installer used very cheap caulk ( $2. ). Buy the best @ $5. a tube and reseal. The cracks are sucking the heat out of your house making it cold and wasting energy. There should only be about an 1/8 of an inch gap, any more and the windows might be ill fitted to your openings.