Builder following unethical practice, delay in possession, not returning full panalty

I have purchased a flat under CLP plan on May 2012. As per Agreement, it should be delivered within 36 months from the agreement date. We have made 95 % payment till March 2017. On April 2017, Builder issued possession letter with 5% Demand note with the promise that Flat will be completed with 45 days from the date of 5% payment. We have visited the site on May 2017 and the flat was not even ready. But we have paid 5 % payment in June month with the assurance of completion within 45 days but the flat got ready in Sep 2017. Now at the time of full and final settlement, builder is not giving us proper penalty. They have deducted 6 months as a grace period from the delayed months and as per them possession letter was issued in April 17 month so no interest will be levied after April 2017. Moreover, they are charging 30000 interest on late payment which is not real as it is a CLP, so Bank has issued payment as per the valuation report of that project. Please let me know if I have to go to consumer court.