S ure, you get a tax deduction for the interest you pay on your mortgage, but that doesn't mean you aren't heartsick about all that interest. Should you try to pay your loan off early and cut out a chunk of the interest due? Broker fees may come in different forms, either as an addition to your interest rate, a separate fee, or points paid at closing. With all these additional fees, you might find that your best mortgage rate deal may not sound so great after all.