My previous company is letting to legal action for joining client company

I have resigned from my previous company and completed all the terms and conditions including the notice period. After that one of the clients contacted me to work with them. This is a client of my previous company and they liked my work so they contacted me. I have joined their company and this client mailed my previous company that I'll be working on an ios app from now on. In reply, my previous company mailed that "Looking forward to work together." Now, after a week I got call from the previous company and they are telling me that I can't work with their client and forcing me to resign else they will take legal action against me. And also saying that it is written in their terms and condition which I signed. But, I don't find such terms written. It is only written that I cannot contact their client which I didn't. I am also adding 2 points below from terms and conditions related to this: 1. We wish your association with company be with integrity, initiative and efficiency. While this agreement is in effect and thereafter, an employee shall not divulge to anyone, any freelance work at home, any confidential or proprietary information regarding the company's records, plans or any other. Aspects of the company's business which it considers confidential or proprietary; provided, an insubstantial or inadvertent disclosure by employee causing no material harm to company is not deemed a breach of this provision. 2. Employee cannot misuse company's property, client information and code. Company's clients, information and infrastructure are the assets of company. In case of employee leaving the company, they cannot contact company's client or use the code of company. Can they take legal action against me on basis of this terms?