If he has done all such illegal activities and is threatening you then you can approach police with a complaint for exorbitant interest and lodge a criminal complaint under the prohibition of exorbitant interest act.
In the police station you can negotiate the amount and close the entire loan amount at one go and get relieved from him instead of suffering the humiliations regularly