Non payment of personal loan dues

I have taken a personal loan on 2011 from a private institution against my salary while i was working with a private company and i have started repaying through ecs regularly. but unfortunately the company was closed and i could not repay by ecs and asked them to stop ecs and collect by cash, initially they did not agree on that and every time i tend to pay bouncing fee for the bank and delay payment and bouncing charges for the lender.it goes like this for many months a large amount of penalty i got to pay, later in the course i lost my other job and i did not get proper job to settle this account. but i sincerely updated them the new contact number change of address loyally and repaid maximum by borrowing money for higher interest from outside to sustain my privilege till 2014-15. one day for non payment of dues a lady called from the lender side and abused me and insulted me unbearably, since then i stopped paying the dues. Now they have started coming home and started abusing which i was told by my wife staying alone with a girl child.Now i am not in a position to repay the balance as i am not in a good position now and came out of city for a job. how can i get a solution for this. and what shall i do. I dont have any proof of abuse from their side.