The toughest restrictions are in markets where home prices are falling, though regions where property ...For new home buyers and those seeking to refinance, it can mean higher down payments and a higher bar for credit scores, among other requirements. This is also perhaps why brokers charge high for closing fees. A percentage of the closing fee you pay on a mortgage goes to the mortgage broker's personal funds. This, along with a few more fees, stands as their salary.