You can issue a legal notice to both the Sri Ram Finance and the seller demanding the return of the amount you have made towards payment to first party at the behest of the second party on an oral agreement for sale of property.
You may state that since the second party is not capable of executing sale deed in your favor due to his personal liabilities with the builder, you are canceling the agreement with the seller hence the finance company to return your money and it is at its liberty to take action for recovery of home loan from the seller/borrower directly
If the demand is not complied with then you can approach court for recovery of your advance amount from the finance company and you can implead the seller and necessary party to the suit.