When I left my previous employer I rolled my 401k account into a rollover IRA. I made a withdrawal from the rollover IRA. I am over 55 and receive a retirement income from my previous employer. If this had been from the terminated 401k I would not have to pay the penalty but since it is from a rollover IRA I have to pay the penalty. What is the difference between the rollover IRA and the 401K? I don't understand why there is a penalty from the rollover IRA but not the 401k.

Some of the rules for IRAs are different that for 401Ks; some are the same.

This is one of the different rules.