I am recently divorced and we have joint custody with me as the primary physical and legal custodian of the children. Does this mean I can claim both children or is he entitled to claim one of the two?

You claim them according to the agreement in the parenting plan ( child custody agreement)
If there is no agreement, then the parent with physical custody gets to claim them.

** Most plans that attorneys do show or state who claims them and when. Mine for example is each parent every other year