But even better deals can be found at smaller banks and credit unions.

"I've found that my clients can get routinely better rates by heading to a more regional lender and forgoing the bigger lenders," says Sean Satkus, a real-estate agent in the Washington, D.C., area.

The differences can be stark. On average, the three biggest banks—Bank of America Corp., Wells Fargo & Co. and J.P. Morgan Chase & Co.—offer rates of 4.66% on 30-year fixed mortgages for home purchases, according to Bankrate.com. By contrast, St. Louis's Heartland Bank is offering a rate of 4.50%. Acacia Federal Savings Bank comes in at 4.25%. And Rockland Trust Co. in Boston is offering just 4.13%. (None of these offers include "points," or extra fees to secure lower rates.)

Jason Schneider

To some extent smaller banks have always been a little more competitive on rates. But "the discrepancy is widening," says Guy Cecala, publisher of Inside Mortgage Finance, an industry newsletter, "and I only expect it to get wider in the future."

Consolidation during and after the financial crisis is the biggest factor. Together, Wells Fargo, BofA and Chase accounted for 56.5% of new mortgage originations in the first half of this year, according to Inside Mortgage Finance—up from just 36.6% in 2007.

Now these banks don't have to compete on pricing in the same way, Mr. Cecala says: "They have a large share of the market, and aren't worried about demand."

Pricing Advantages

Smaller mortgage brokers and regional banks also have some built-in pricing advantages, says Greg McBride, a senior financial analyst at Bankrate.com. They are nimbler than larger competitors, he says, and can cut overhead when they need to—making them lean enough to price loans aggressively.

Compensation structures for loan officers are also a factor. Smaller lenders tend to pay on commission, "and will sometimes operate on thinner margins to get higher volumes of loans out," Mr. McBride says. "Mortgage brokers tend to live or die on volume." By contrast, big banks tend to pay loan officers a salary regardless of volume. In fact, it often is cheaper for big banks simply to buy loans originated by smaller banks, or "correspondent lenders," than to reduce rates to compete with them.

"The bigger banks can save money on origination by buying the loans from correspondent lenders later in the chain," Mr. McBride says.

Profit margins are falling sharply at smaller firms. According to a July 20 study by the Mortgage Bankers Association, profits per origination at independent mortgage lenders were down to $606 during the first quarter of 2010 from $1,088 in the same quarter last year.

Room to Fall

Yet there still may be room to fall, says Cameron Findlay, chief economist of LendingTree, an online lender. On Aug. 1, he says, the average rate for 30-year fixed-rate mortgages was 4.56%, which was more than one percentage point greater than the average of 3.45% at which lenders could sell these loans to investors in the secondary market. "There's clearly more wiggle room there," he says.

Borrowers looking for smaller lenders or brokers can trawl websites like Lendingtree and Bankrate.com, which lists rates in local regions.

Alex Sorokin is taking advantage of small banks' largess. The 52-year old accountant wanted to buy a two-bedroom condominium in Brooklyn, N.Y., and initially thought he would get the best rate from Chase, Commerce Bank or Wachovia, now part of Wells Fargo.

When he started inquiring in April, he says, he was surprised to find that none would offer a rate for a $279,000 mortgage lower than 4.8%. That is when he decided to contact Luxury Mortgage, a Connecticut-based lender. He ended up with a rate of 4.625%. 

"We closed in July," Mr. Sorokin says, "and I am really happy."