It's not out of the question but there are a lot of options you can explore before you take a step like that. You can look into the possibility of getting a loan modification to lower your payments or refinancing your first mortgage to pay off the second.
Linear Mode
