I am 43 years old. In 2007 I took a hardship & removed $20,000 from my 401k. $2.000.00 was taken for taxes. I applied the rest in the purchase of my first new home. Now that it is time to file for tax season, will I have to pay taxes on the $18,000 that I used as a downpayment on a home loan? Does the IRS take in account that it was used to purchase a home?