We are removing a chain link fence. Should we leave the concrete in the ground? If no, how do we get it out?

Dig out around every post till its loose and pull it up. If you decide to leave the concrete just take a sawzaw and cut below the ground and just remove the pipe

No, don't leave concrete in the ground.
Dig along one side of the concrete.
Exert leverage against the fixed side of the ground until post and concrete is lying on it's side.
Break concrete from around post. You might need a hammer drill if doing a lot of posts. Post with concrete may be to heavy to move.

Well , I don't personally see how you would remove the posts, without the concrete. So dig around it until it is loose enough.

After you have gotten the posts and concrete out of the ground, fill with dirt. Then in a couple of months, go back and refill the holes to take care of any settling dips.