Homebuyer and builder despute

I have bought a house from a builder, he at first stated and says he is the best builder in city and built more than 100+ buildings and we trusted him, and made a deal where along with land and construction of the house total value is 55 lakhs, everything is fine (note he is builder not owner). The builder when I came to check the home the quality isn't good and I argued with him at that time in 2020 but he didn't listened my words and continued his work where I have paid 40 till now, but after seeing the work and house I stopped paying him the rest amount. Due to he didn't constructed well. I said rather giving you the rest amount (15lax) I prefer to build, because the building is built by low quality along with doors, walls and everything got cracks. I also enquired two to three builder recently they said the building actual cost was 40 lakhs. "the builder cheated you and thought gaining 15 extra amount." He replied. Now I raised the same with the rest members I want to appeal this towards court. He didn't gave me possession but I darely entered into my house. Is this wrong step? ( Land is registered on my name. I even taken a bank loan on the house). How to legally proceed this without paying single penny to the builder? I have consulted a lawyers but he is buying all of them.