I have a rollover 401k from a prior job that is completely separate from my current work 401k. I would like to use the money from the prior rollover 401k for a down payment on a house. Can I take the entire amount? I know there will be a penalty plus taxes and am willing to pay that. I know that a 10,000 withdrawal there is no penalty, but say I took 40,000.00 would I be paying a penalty on the entire 40,000 or would 10,000 of withdrawal be free of penalty. Also, how is this paid back? What I mean is, this money has nothing to do with my work, do I still have to pay it back even though I'm using my own money?