Builder not handing over Maintenance

Of 138 residents of a housing society, in Kolkata, a few of us (22 in a case and 45 in another) filed a civil suit against our builder for a) breaking a wall separating the commercial area and residential area in the same building b) for adding newly constructed building in the complex by demolishing a boundary wall, where by reducing paid up common area and sharing of resources. case a) went up to High court where the injunction given by the lower court was vacated and HC instructed lower court to expedite completion of case as per case merit. Now since the injunction has been lifted builder has started all activities inside the wall. Not only this even after 3 years when builder did not hand over maintenance, we the residents took it over deploying our own security & other service people and redirecting the fund from builder to our own society. Earlier we had written that we had formed a society independently and wish that builder completed his formalities of handing over and registering the society as per West Bengal housing society act and builder should produce the accounts statement of income and expenses and the corpus fund he had collected. He shot a lawyer's notice barring us from forming an association and threatened us of legal consequences if we did not take back our words. As of now the injunction of case a) & b) has been vacated and a 3rd case in consumer forum (state level) is on. Builder is now threatening to sue us with defamation and loss of business opportunity. He has singled 3/4 people who are elected members of the association. This association has been registered under society act. My question is where do we stand? Can the builder really sue us? Is there no stipulated time span by which the builder is bound by law to handover. Is our taking over after 3 years and running the maintenance for 1 year illegal?