Dude threatening legal action against me for email - please help (mumbai)

So this guy wanted a service from me and for one month he kept dillydallying and making laaaarge phone calls. Finally a few days ago we hashed out payment terms and I sent him details and asked him to pay by that evening otherwise I would need to cancel his appointments as my time slots are limited. He then made an about-turn and said "I am a bit surprised by tone of your message and find it demanding. Looks like we have hit an impasse. You are free to act as you deem fit." This dude had a whole spiel about how his mum died of cancer and he had bills of 7.5 lakh every month on an income of 2.5 lakh and how he values relationships because he is a Gujarati and blah blah blah. Like we're talking half an hour phone calls a couple times a week. This dude had wasted quite a bit of my time and I was upset. I shot off an email: https://old.reddit.com/r/india/comments/chmbgx/dude_threatening_legal_action_against_me_for/ (sorry I can't post the whole email here) basically there was NO name calling or slurs or ANYTHING!! Here is what I got back: "Your personal remarks against me are derogatory, and uncalled for. Needless to say, this has drawn my lawyers attention to pursue legal recourse under civil proceedings. I advise you to unequivocally apologise for your distasteful remarks." What should I do? I honestly do not think it's derogatory and afaik there is nothing legal he can really do I mean, what? I didn't even curse. Sounds ridiculous to me. FWIW I'm a woman and he's a man. What should I do or say?