Joint venture

I have entered into JV with a builder to build 6 flats on my property where 2650 sqft. the building was demolished for the purpose. At first, he agreed to build flats later on changed his stance saying flats could not be sold and changing the plan to build 3 Villas out of which one for us and 2 for sale. We took Rs.25 lacs towards the loan on the property and closed the loan. We chose to take one Villa without seeing the other 2 villas. Since ours is a very narrow one we asked him to give us another villa which he refused stating that he has taken advance for the 2 villas. Actually, the building plan has yet to get Corporation approval and no preliminary work started on the land. Since he refused to give us the requested villa, we wanted to go for some other builder and return the money we had taken to close the loan. The project is yet to start even though we have given possession of the property eight months ago. Now he demands the money along with interest and other expenses he has incurred. The 8-month delay is his change of plans from the original JV for flats and now for the Villas. He threatens to file law suit in the court. Please advise how to proceed.