I removed ceiling fan and the box had a red, white and black wires coming out. The light fixture only has two wires and a ground how do I install it?

There must have been 2 switches for the fan, one for the light, one for the fan. Yes? Pick one, red or black and use that for the fixture black, white to white, ground to ground, cap off other lead but leave in ceiling in case you want a fan again. One of the switches will work the light.

In the box two black were joined together and two white and then the red. The switch that worked the ceiling fan has a red and a black wire attached to it.

In the box two black were joined together and two white and then the red. The switch that worked the ceiling fan has a red and a black wire attached to it.
Here's what I would do. Hang the light fixture by a wire. Connect the fixture black to the ceiling red and the ceiling white to fixture white. Try the switch. If it works, wire nut the blacks and hang the fixture. If it doesn't work, disconnect the red, connect the ceiling black to fixture black and try. If it works, wire nut the red and hang fixture. One of the black or red has to be a switched hot, then you will have a complete circuit with the ceiling white.