I have been a parishner at a church for 4 years. An attorney who recently came is representing my ex wife in a custody issue, against me. Is this wrong or a conflict of interest. How do I handle this with pastor and with church. It is making me want to leave this church?

Is this wrong or a conflict of interest. - No and why should it be as there is no professional misconduct here at all.
Your only link is the church you both attend.

Its nothing personal against you... its his JOB. If it puts you in an uncomfortable position then you may need to find a different place of worship. Don't make it awkward for him. Or, you could man up to the situation, be mature about it, and move on with your life as Christ would have you do.

How about shaking his hand, and saying Welcome? Are you doing something he has reason to spy on? Of course not. Take the high road, and who knows, you might like him, and he might like you. And it might even be to your benefit in any disputes.

Many lawyers are good buddies with their opposing lawyers, outside the courtroom. Many lawyers can't stand their clients, and vice versa. It's a job.

It could pose a conflict if the attorney is using the connection to dig up dirt in you or infiltrate your social circle. Much depends on the size of the church. I attend a large parish where avoiding someone is quite easy, but I have friends in churches so small and intimate it is an extension if family. Talk to your pastor about your concerns. Just being at service with you, not a conflict. Showing up at the churches "divorce support group", huge issue.

It depends. Is he coming there because of you or because he likes it? If he's coming there to investigate you, yeah, that could be a bit of a problem. But if he just happens to be attending the same church you do, so what? Follow Joypulv's advice and get to know him with no preconceptions. You might be surprised.