Removing ex husbands name off of mortgage

i was recently divorced and in our divorce decree i am to refinance my home if i got remarried. i recently got remarried and have tried to refinance to get his name off of the mortgage, but i have not had any luck at this time due to the fact that i have all the debt in my name, and also paid out my ex the equity at the time of our divorce. he is now threatening to take me to court if i don't get his name off, so he can get a loan for his own home. he says if i don't the court will make me sell. the property is in my name, and he has been paid out his equity, so he has no financial interest in the home. do you think this will happen? i don't know what to do!!!!!!!!!!!