Notice period issue

Hi, I just had a simple query regarding the notice period in any private firm. What happened is that before resigning I had a discussion with my manager (verbal discussion) regarding reduction of the notice period just by a few days. Say it was 90 days and I wanted to reduce by couple of weeks. Now the policy states that a manager can relieve someone at his discretion or the company can come to a financial understanding wherein the employee pays a penalty equal to his gross salary for each day he is relieved early. Verbally it was agreed to be the former (managers discretion option). And no where was penalty mentioned. The penalty amount is usually when the new company agrees to a notice period buy out. But here there was no new company. Just me leaving early. Anyway, so on the last day I get a message saying that I have to pay the full amount penalty for the no of days I am leaving early (as per my gross salary). So I called up the manager and he said he will look into it. HR said don't worry there is a way around it. There must be some error in the request that was raised to relieve me, etc... Now they are essentially trying to extort a massive sum from me saying that I have to pay it. Whereas I am telling them that I never agreed to such a thing. The only reason I decided to leave a few days early was because my Manager assured me it's ok. Apart from this, I had an equal amount of earned leaves which I had not taken, have worked very hard for the company and this is how they are treating me. And they are not releasing my experience letter. That letter is very important from a career point of view. It's been months and I am stuck and they say they won't release my experience letter. So I just wanted some advice about my legal position on this ? Any advice will be appreciated. Thanks in advance