Friend not returning money and threatening me in return

Sir, Once I took some amount from a friend of my Spouse and I returned him within his stipulated time and without any follow-up from his side. Later, he got some urgency and asked me for the amount and I was not in a good position but as a token of Gratitude, I offered him to use my credit card for urgency. He used my card for Rs.50000/- which in turn debited my Card for Rs.51200/- with charges as he transferred amount to wallet. As I earlier haven't paid any interest for amount I borrowed so I asked him to payback only Rs.50,000/- but, he did many excuses since 3 months and he promised me earlier that he'll return amount within 15 days. In recent times, he started shouting on me and my wife and threatening me to come to his city if I want amount. He was threatening my wife on call to shut her mouth and used a wild tone. As she recorded everything, I was able to listen all the things so I asked her not to call him again. Now I wanted to go legally with this issue and recover my amount with Bank rate for 2 months (leaving one month for his past help) and legal charges thereof. Please guide me accordingly.