Search:

He's not a good father but you are OK with him seeing them. Isn't that a contradiction?

You say he never did anything for them, but helps provide a roof over their heads and food on the table.
...

Ok, so its not that he won't let you keep the children, but that he won't let you take them from him out of the country. And I'm assuming that its not just for a visit but when your visa expires.
...

By partner I assume you are not married. Marriage may let you remain in the UK.

What do you mean by "keep the kids"? Do you not live together? Is he not allowing you to take them with you when...