My ex and I finalized the divorce in 2003. I have full custody and he gets supervised visits with them. It has been 3 1/2 years since he has seen the kids. He was very sparodic in support payments and gave me impression he wouldn't file his tax return cause he would owe a lot. I claimed them for the last 5 years. He now is saying he has the right to claim. I check the divorce paper and it says he could claim them but at the time of signing the divorce paper the child custody was pending on court case involving the kids. From my reserching I qualify to claim them because they are with me most of the time which is this case is everyday of the year. Please help me figure out what to do to be sure that I can and should be able to legally cliam them for the years onward.

If the court papers state he gets to claim them, than that is how it is. Unless there was some sort of stipulation written in... he can claim them only as long as he is current on support payments, for example.
You do qualify according to the IRS requirements of who provides the majority of care, but the court order overrides that.
Since there was some sort of issue regarding custody... find out what the final documents that were filed contain.