It is illegal. I looked up the age of consent laws for Australia. The ages vary by state/territory but the youngest age a person can legally give consent in any of them is 16.
Age of consent laws - Child Family Community Australia
The only defense he might have is if he believed you were older than you are. But he is still the supposedly responsible adult.
I am not saying you have to go to the police. But you are not responsible for what a grown man did. No matter what anyone tells you, the shame lies with him, not you.
At the very least, you need to inform your parents or another adult who you trust. This is not something you should go through alone.