Builder Harassing us

Hi Team, My name is Vijetha and i am from Bangalore. We are three of us in a family. My dad, mom and myself. We signed up with a Builder to construct a House of two floor in our plot on February last year and yet he has not finished construction and its in the final stage now but he blackmails us to give entire money to complete the building if not, he says he will not finish it till we pay entire money. In the contract he mentioned he will handover the building within a year but by word he said he will finish it within 6 months. He claims that he has been in trouble because of demonetization so he has no money to complete our building. We don't believe him so we dont want to pay entire money as he might not finish pending work and might leave in mid so we want to keep the balance money as a security. When handovers a keys, we want to check if all is fine with the quality of the work and then pay him his balance money. We agreed for 38 lakhs and we have payed 37 Lakhs till now. Just for 1 lakh he is harassing us. Please guide us.