Hi
it is necessary now you should notify him first and if not obliged, then send legal notice for his unjustified actions which caused you loss.
You should bring it to the notice of the builder that the DD has been accepted by him, so there is no question of penalty. Regarding the the increase amount if you have objection confront him of that actual agreement/contract varies from his present claim of increased charges.
You can approach the consumer forum with your grievances and seek compensation for delayed payment n terms of rent if not paid and financial , loss and mental harassment.
if you think it is necessary that you get the possession , proceed against the builder for the compensation of all his unfair approaches and charges, delay etc after getting the possession. Even after getting the possession, you should move too a consumer forum against him for his delays and his unjustified trade tactics.