Do Not worry such agreement will have no effect.
Forfeiture of entire amount on cancellation is agreement is not permissible under agreement even if one of the aprties under forceful circumstances signed the clause containing such forfeiture clause.
Sp send him a legal notice claiming refund of your earnest money.
Once the builder refuses to doso file a case before the consumer forum wherein you will get not only the refund of money but damages and compensation in addition tot hat.