Ok, two cars, the one at stop SIGN first has the right of way.

My question is why did you sit there the entire time he was stoping and seconds latter.

Most likely they thought you were waiting...