Owner/Broker is asking me to evict the flat before lease expiry.

I have rented an apartment in Mumbai, I'm paying the rent to owner from time to time, with absolutely no delay. I have also given deposit and paid a month brokerage to the broker. Everything was fine for 4-6 months. Suddenly, one day broker called up and said that owner wishes to sell the flat, so we will be bringing customers to the apartment for selling purposes. I agreed to same, showed no objection. I'm a working professional in MNC, I have to travel multiple times, so many a times I'm not at home(i.e. rented apartment), due to which the broker/owner is not able to bring customers to the apartment, due to which the broker is now pissed off and asking me to evict the property. I have already said, I still got 2.5 months left as per agreement. Broker is saying, if you don't evict the property, I will give you notice then you will have to evict the property at earliest. Broker was also asking me to handover duplicate apartment keys, so that he can bring clients to the apartment in my absence as I'm frequent traveller and not present at home, which I absolutely denied handing him the duplicate apartment keys as it will breach my privacy. On this denial, broker is more pissed. Kindly suggest.