If you have been in the house for a few years now and you have built up ...As a first time home buyer you can not always get the best rate. Maybe you did not have twenty percent to put down or maybe your credit needed a little rebuilding. Generally, the lower the interest rate, the more points the lending institution will charge.