Need guidance :house construction dispute with contractor

Hi, I recently built a house by giving a material contract to a contractor.This person is not a professional contractor but a working employee in TCS Admin at some manager position.He stays in same layout as I do and he wooed me saying I have build many house in Bangalore and have good experience and starting this new business/alternate career option of building houses.I trusted him because I thought he is from same layout so won't cheat me.There was an agreement signed and he made a agreement in his wife's name and he being point of contact.Soon after 15 to 30 days of construction being started he started torturing me by putting restrictions as this cannot be done, that would require extra money and back tracking his oral promises.For each stage(milestone payment) the money taken in advance and I did not want to distrust by house construction so was adjusting his tantrums and going on.Towards the end of my house he put an additional demand of 1.87 lakh rupees for roof to roof measurement of which we negotiated it to 97 thousand rupees.This was a surprise element as we never asked for any extension,neighter did he communicate to us that there is some extension he is doing.And there was a clause in agreement which said roof to roof measurement would be done towards end of house construction, so he made use of that clause.There was another 1.3 lakh rupees we had to pay extra for items he back tracked or was putting of inferior quality and we intervened and said we want a min of those items.Overall at 1550/- sqft we went to agreement he has taken around 2.3 lakh extra above the agreement amount(60 lakhs) and provided a house with lot of issues.There is a 88 thousand rupees which is to be paid after 3 months of handover.Now the problem is he refused to correct issues shown in house and he has spoilt most of my doors, he has put broken granites in house and the entire house needs one coat of painting, the parking tiles are damaged etc.Now you may ask why did you entertain him so far and why did you keep giving him money.The problem is it took me a while 1 month to 3 month to get to know that this guy is a cheat and doing substandard work.But each time about 3 to 4 lakh of my money was stuck with him and if i stopped work then it would cost me more to finish the remaining stuff plus this guy would say he has laready put money from his side and claim extra money.Towards the end he even threathed me if i dont pay him extra money he won't handover the house to me so i had to pay that extra 2.3 lakh above all this he handed my house in bad condition.I need suggestion as to how do I proceed with such person, and how can i utilise the retention money to correct these issues.He is going to come 3 months from now and said if I dont pay him the retention money completely he will use his contacts with some rowdies and local MLA and recover it from me.I never thought a person working for an IT firm like TCS in Bangalore can be issuing such threats like a local contractor.So I would like to know what are the options with me ? I have moved to the house and plan to start interior work before which 1.I have taken photos of damages do I need to do anything else ? 2.Is there any govt evaluator who can come and estimate the damages and hold proof in court in case this issues goes that way ? 3.Is there any govt photographer who can come take video of the damages and that can act as proof when this person comes to ask for retention money ? Should I hold my interior work plans till the issue is resolved or can i proceed with it after taking photos ? In case this person threatens or send some goondas what should be my course of action ?.Most of my communication has been over email with him and I have signatures of the done payment with his wife for each stage.Need proper guidance to at least save that retention money and utilize it to correct the damages of my house[BTW there is clause in agreement which says in case the builder does not correct the damages in 15 days those can be corrected by owner and money be deducted from retention money]