Moving in together and holding yourself out to be the child's father are two different things - if you presented yourself, introduced yourself, as the child's adoptive father, perhaps signed documents or permission slips as the father, then you held yourself out to be the father (adoptive or otherwise).
It is the same with common law marriage - you MUST present yourself as a married couple. You don't just live together.
Again - I'd consult with an Attorney and hope you'll come back and let us know what he/she says, how this works out.