My tax return was rejected by the IRS. I discovered the error and corrected it. I also discovered another error and corrected it which also changes the amount of taxes I owe. Am I allowed to make the second change which was not the reason for the rejection. Will I be penalized for being late since I made the additional correction.

If your return is rejected, that means that it is not filed. So you can make any number of changes.
If your return has been accepted, then you can not submit corrected return. Instead you must file amended return Form 1040X. If you owe tax on the amended return, you will pay with interest and penalty. If you do not owe tax, you will not be penalized.