I would call the IRS and inform them of what he is doing. Only the person who supports the child for more than half the year and has the child living with them for at least 6 months of the year is legally entitled to claim them. If you call the IRS they might be able to flag his ss# so they can stop him if he even tries.
If he does beat you and they say ss# has been used you will probably have to wait for your whole refund. About 7 years ago it happened to my mother with my little sister, no one else claimed her, but the IRS was just doing a random EIC audit. Her whole refund was held up until they were satisfied, so that is probably what would happen.
Even if the IRS can't flag him or anything like that, maybe they can tell you what you can do.