6 years ago had my tongue pierced, no problems; about 3 months afterwards, I used a clear spacer w/clear small rubber band to keep it open---well, the piercing got extremely sore and when I went to take the spacer out, it seems that the clear rubber band either got stuck while taking the spacer out, or the tongue healed over the rubber band... leaving a bump and sore spot on the under side of my tongue. I went to a Dr. and he couldn't find anything... and said it would "work itself out eventually" and gave me some antibiotics to take. Well, 2 days ago I started to get soreness under my tongue and woke up this a.m. with swollen tongue and very sore bump, and pus coming from the underside hole where the piercing had been. The top of the tongue is fine, no redness, no pus; just the underside. So what do I do? If the rubber band is in there, will it come out, or will I continue to get infections the rest of my life because I have a 'foreign part' in my body now?

You should go to the emergency room. Tell the ER doctor about the rubber band. This will likely cause recurrent problems until the rubber band is removed.

Its probably too late to answer but the same thing happened to me.
I went to bed with my clear rubber tongue ring band in my mouth the next morning when I woke up I was in so much pain! I discovered the clear bar still in my tongue with the rubber top inbeded in my tongue and the skin in a sense covered the top of my tongue.. and as stupid as it sounds I pulled out the bar. Leaving the top still inbeded in my tongue.. I was too scared to go doctors as I was only 16 at the time.. so I used a metal tongue ring and somewhat pushed out the rubber top in my tongue.. this was very painfull but successful. It came out after some pain and abit of blood but it felt so good to not have that lump in my tongue or pain as its very noticeable.

In your case it sounds as if you let the hole grow out.. so I'm not sure what to do.. as I still has a slight hole in my tongue when I got the top of it out.

But yeah note to anyone with a tongue ring.
Never go to sleep with the plastic rubber ones that bend in your tongue at night!

I have this now I'm to scared to go to the doctor it to pain full to pull out but its been like 2 days now and I feel like the skin is just growing more over so ill suck it up and pull it out later