If I eat frozen foods, like frozen mangos, am I actually burning calories digesting the food?

Not enough to matter... over unfrozen ones anyway. The amount is essentially insignificant.

You do burn some but not a lot, eating or drinking something really cold makes your body work to warm it up for you.

you do burn some but not a lot, eating or drinking something really cold makes your body work to warm it up for you.
If the body has to work harder to digest frozen food, then yes more calories are being consumed, therefore metabolism is stepped up. So yes, more calories are burned.