I've always been a little overweight, and I don't exercise at all, and I was just wondering how fit you have to be to join.

Run 5 miles within a few weeks, do 100 push up and sit up and so on.

So start exercising

Army Physical Fitness Test (APFT)

You technically don't have to be fit to join. That's what BCT is for, but you'll have a much better time if you are fit coming in. If you want do something like special forces (Green Berets, Rangers) you have to be ridiculously fit going in.
Swim 500m in under 8 min
100 push ups in 2 min
100 sit ups in 2 min
25 pull ups straight
Run 1.5 miles in under 8 min
Those would be considered very good statistics going into RASP or SFAS.