We recently cashed in our 401 to pay the house off. Are we responsible for paying
Federal and State Income tax on the total amount of the withdrawal.

You should have been given paperwork that explains your tax liability and the penalty fees associated with early withdrawal.

Since you already did this, its too late to tell you what a dumb move that was.

Instead of having a tax deduction for your mortgage interest, you now have to pay taxes on your distribution PLUS a 10% penalty. That 10% will probably wind up costing you WAY more than the interest you saved by paying off the mortgage.

So the answer is yes. Unless you were over 55, you will have to pay the 10%, Even if you were over 55 you might still have to pay it. As for the taxes, you will need to add the amount of your distribution to your taxable income for the year of distribution and then pay taxes on your total income.