I had a tooth removed and there is what looks like a piece of the tooth still there. It is sharp and sensitive. Do I need to go back to the dentist?

It could be a bone chip. You could go back to the dentist but if it's a chip it will work it's way to the surface for you to be able to remove it with clean tweezers. If you have a denture resting on the area, go back to the dentist and have it removed.
You could check with the dental office to be sure the dentist didn't break the tooth on removal (he/she should have shown you the tooth, roots and all).