Torture from Money lender

Hi Sir, I am geeting huge torture from money lender from my home town. I have taken 5 lakhs for interest from him. Within 2 years i have paid 4.2 lakhs interest itsef. On may 5,2017 i have paid maximum amount by taking personal loan. There was a shortage of 88 thousand. That 88k also pending interest only. I have requested to consider this as actual amount. After repaying that amount i was in deep financial crisis and paid 1.08 lakh rupees recently. My Friend is the mediator. I clearly told him to close that and asked to give back my promissory notes and cheques.Since i am in out station i trusted my friend a lot. now a days again money lender directly came back to me with new interest to pay 1.10 lkh again.. This is really unfair. I wanted to go legally. Since my cheques and promissory notes are there with him, is there any negative impact on me? I am concernning my job also if i fall into any negative cases created by him. I have done all transaction through Bank only. But he always gave some commercial accounts to send. Always they are threatening me that they have notes and cheques. Kindly help me how to proceed.