For 2006 and 2007 I claimed my son on my taxes. I had to do both years at the same time because of W2 problems, but that was resolved. Now, the mother has told me she is contacting the IRS and reporting me for fraud. She is the custodial parent, but I have several months worth of receipts from Apr. 2006 (birthdate) to January of 2007 proving the items I provided for both of us to have the child, diapers, food, medicine, etc. Although we were never married, I kept visitation with my son, because I love him. During those first months, I would have him a week at a time, and trade off every other one, so it adds up to about half the time. I was ordered to pay child support, which I have been currently doing ever since. My back child support was set even though I had receipts, because I wanted her to have extra money, seeing as she has never been employed with any company in her entire life. I made payments on both, and when tax time came around, they withheld the back support, thankfully. Now, she is claiming tax fraud on me, saying she has proof that she has custody of him (which she does, of course) and that I misfiled. I still see my son every weekend to this day, and faithfully pay my support. Should I have something to worry about?

From what I understand you should be able to claim him IF she is not working and claiming him herself.
Him not living at your house may be the problem.
BUT I always hear separated couples with kids discussing who is going to claim which kid on the tax return this year.