With modern high divorce rates, a lot of people get married more than once in their lives. Saying my ex-wife's new husband is a bit awkward. Is there anything more graceful?

Conversationally, i agree that ex-wife seems much more common that former wife. In writing, though, the use of former doesn't seem so rare. Here's an interesting ngram.