I know that property taxes paid on my home are deductible when I itemize my return, but if I pay a penalty for making the payment late, can I deduct the penalty as well? Thank you.

No.